520 R6c'tfPC7/PTO 08HQVIQQQ 



FORM PTO 1390 US DEPARTMENT OF COMMERCE PATENT AND TRADEMARK OFFICE 
(REV 5-93) 

TRANSMITTAL LETTER TO THE UNITED STATES 
DESIGNATED/ELECTED OFFICE (DO/EO/US) 
CONCERNING A FILING UNDER 35 USC 371 


ATTORNEY DOCKET NO. 
2000J520A 


U.S. A£PUCATION NO. 


International Application No. 

PCT/JP99/02385 


International Filing Date 

May 7, 1999 


Priority Date Claimed 

May 8, 1998 



Title of Invention 

SOLID-STATE COLOR IMAGE PICKUP APPARATUS 



Applicants) For DO/EO/US 

Hiromasa MIZUKI; Masaaki YAMASHITA; Yasushi KIYOSHIGE and Hidehiko WATANABE 

Applicant herewith submits to the United States Designated/Elected Office (DO/EO/US) the following items and other information: 
L [X] This is a FIRST submission of items concerning a filing under 35 USC 37 1 . 



2. Q This is a SECOND or SUBSEQUENT submission of items concerning a filing under 35 USC 371 . 

3 - [XJ This is an express request to begin national examination procedures (35 USC 371(f)) at any time rather than delay examination 

1' until the expiration of the applicable time limit set in 35 USC 371(b) and PCT Articles 22 and 39(1). 

4. fX} A proper Demand for International Preliminary Examination was made by the 19th month from the earliest claimed priority 

"'z date- 

5. [JCJ A copy of the International Application as filed (35 USC 371(c)(2)) 

a- [] is transmitted herewith (required only if not transmitted by the International Bureau). 

b\ [X] has been transmitted by the International Bureau. - Attachment "A" 

' =;«. [] is not required, as the application was filed in the United States Receiving Office (RO/US). 

. 6. |XJ A translation of the International Application into English (35 USC 371(c)(2)). - Attachment "B" 



7. Q;: Amendments to the claims of the International Application under PCT Article 19 (35 USC 371(c)(3)). 

a- [] are transmitted herewith (required only if not transmitted by the International Bureau). 

,b. [] have been transmitted by the International Bureau. 

c. [] have not been made; however, the time limit for making such amendments has NOT expired. 

d. [] have not been made and will not be made. 



8. □ A translation of the amendments to the claims under PCT Article 19 (35 USC 371(c)(3)). 

9. [X] An unexecuted oath or declaration of the inventor(s) (35 USC 371(c)(4)). -Attachment *'C" 

10. [] A translation of the annexes to the International Preliminary Examination Report under PCT Article 36, (35 USC 371(c)(5)). 

Items 11. to 16. below concern other document(s) or information included: 

11. [X] An Information Disclosure Statement under 37 CFR 1.97 and 1.98. - Attachment "D" 

12. [] An assignment document for recording. A separate cover sheet in compliance with 37 CFR 3.28 and 3.31 is included. 

13. [X] A FIRST preliminary amendment. -Attachment "E" 

0 A SECOND or SUBSEQUENT preliminary amendment. 
14. 0 A substitute specification. 

15, [] A change of power of attorney and/or address letter. 

1 6 . [X] Other items or information : 

Form PCT/IB/304 - Attachment "F" 



529Rec'dPCT/PTC 08 NOV2000 



U.S. APPLICATION NO. (if, 
NEW n ' 



lATION NO. (if Jmown, * 3MI|1.? 

09/ 674 9?? 



INTERNATIONAL APPLICATION NO. 

PCT/JP99/02385 



17. [X] The following fees are submitted 

BASIC NATIONAL FEE (37 CFR 1.492(a)(l)-(5)): 

[X] Search Report has been prepared by the EPO or JPO $860.00 

0 International preliminary examination fee paid to USPTO (37 CFR 1.482) $690.00 

0 No international preliminary examination fee paid to USPTO (37 CFR 1.482) 

but international search fee paid to USPTO (37 CFR 1.445(a)(2)) $700.00 

D Neither international preliminary examination fee (37 CFR 1.482) nor 

international search fee (37 CFR 1.445(a)(2)) paid to USPTO $1000.00 

D International preliminary examination fee paid to USPTO (37 CFR 1.482) 

and all claims satisfied provisions of PCT Article 33(2)-33(4) $100.00 

ENTER APPROPRIATE BASIC FEE AMOUNT = 



CALCULATIONS 



[ PTO USE ONLY 



Surfcfiarge of $130.00 for furnishing the oath or declaration later than Q 20 Q 30 months from the 
eartijit claimed priority date (37 CFR 1.492(e)). 



Muiiple dependent claim(s) (if applicable) 



TOTAL OF ABOVE CALCULATIONS = 



Reduction by 1/2 for filing by small entity, if applicable. Verified Small Entity Statement must also 
be fifed. (Note 37 CFR 1.9, 1.27, 1.28) - 



SUBTOTAL = 



Processing fee of $130.00 for furnishing the English translation later than 0 20 Q 30 months from 
the earliest claimed priority date (37 CFR 1.492(f)). + 



TOTAL NATIONAL FEE = 



Fee for recording the enclosed assignment (37 CFR 1.21(h)). The assignment must be 
accompanied by an appropriate cover sheet (37 CFR 3.28, 3.31) ($40 per property). 



TOTAL FEES ENCLOSED = 



$1,004.00 



Amount to be 
refunded: 



A 



09/674933 
529 Rec'd PCT/FTC 08 H0V?m 



a. [X] A check in the amount of $ 1,004 to cover the above fees is enclosed. 

b. Q Please charge my Deposit Account No. 23-0975 in the amount of $ to cover the above fees. A 

duplicate copy of this sheet is enclosed. 

c. [X] The Commissioner is hereby authorized to charge any additional fees which may be required, or credit any 

overpayment to Deposit Account No. 23-0975 . A duplicate copy of this sheet is enclosed. 

NOTE: Where an appropriate time limit under 37 CFR 1.494 or 1.495 has not been met, a petition to revive 
(37 CFR 1.137(a) or (b)) must be filed and granted to restore the application to pending status. 



SEND ALL CORRESPONDENCE TO: 

WENDEROTH, LIND & PONACK 
2033 K St., N.W., Suite 800 
Washington, D.C. 20006 

Telephone: (202) 721-8200 
Facsimile: (202) 721-8250 




Nils E. Pedersen 
Registration No. 33,145 



November 8, 2000 Check No. 

NEP/kjf 



THE COMMISSIONER IS AUTHORIZED- 
TO CHARGE ANY DEFICIENCY IN THE 
FEE FOR THIS PAPER TO DEPOSIT 
ACCOUNT NO. 23-0975. 



-3- 



09/674933 

529Rec , dPCT/PTC08NOV;000 

IN THE UNITED STATES PATENT AND TRADEMARK OFFICE 
In re application of 

HiromasaMIZUKIetal. Attn: BOXPCT 

Serial No. NEW Docket No. 2000-1520A 

Filed November 8, 2000 

SOLID-STATE COLOR MAGE PICKUP APPARATUS 
[Corresponding to PCT/JP99/02385 
Filed May 7, 1999] 



PRELIMINARY AMENDMENT 

M COMMISSIONER IS AUTHORIZED 

Assistant Commissioner for Patents 10 CHAR GE ANY DEFICIENCY IN THE 

Washington, DC 2023 1 F£E F0R THI $ PAPER TO DEPOSIT 

5 ACCOUNT NO. 23-0975 



Prior to examination of the above-identified PCT application, kindly amend the application 
as follows: 

IN THE SPECIFICATION : 

Kindly amend as follows: 

Page 6, 

lines 3 to 4, change "two kinds of chrominance data each by one" to --two kinds of 
chrominance data each by one, two, or four--. 

Page 8, 

line 14, change "color-difference signals" to -chrominance signals--. 



lines 16 to 24, change "the signal processing circuit generates two of the four luminance 
signals by using only the data from the whole-color-pass filters, and generates the remaining two 



luminance signals by using the data from the whole-color-pass filters and the data from peripheral 
pixels of the four pixels adjoining vertically and horizontally, and generates the two kinds of color- 
difference signals by using the data from the cyan or yellow-pass filter and the data from the 
peripheral pixels." to -the signal processing circuit generates, among the four luminance signals, 
two luminance signals in the positions of the two whole-color-pass filters by using only the data 
obtained from the positions, and generates the remaining two luminance signals in the positions of 
the filters transparent to two different kinds of colors, by using the chrominance data obtained from 
the respective positions and the data obtained from peripheral pixels of the four pixels, and generates 
the chrominance signals by using the data obtained from peripheral pixels of the four pixels including 
the data obtained from the positions of the filters transparent to two different kinds of colors - 
line 25, after "the whole-color-pass filters," insert -and the remaining two luminance signals 
are generated by interpolation from the data of peripheral pixels including the chrominance data in 
the positions of the filters transparent to two different kinds of colors,-. 

Page 9, 

line 8, change "color-difference signals" to -chrominance signals-, 
line 19, change "color-difference signals" to -chrominance signals--. 
Page 10, 

lines 1 to 6, delete "the color separation filter having one arrangement pattern comprising 
vertically and horizontally adjoining four pixels, comprises a whole-color-pass filter and a cyan-pass 
filter for the upper two pixels from the left, and a yellow-pass-filter and a whole-color-pass filter for 
the lower two pixels from the left, and". 

line 15, change "a whole-color-pass signal" to —a luminance signal--. 



Page 14, 

lines 2 to 3, change "the color-difference signal" to —the chrominance signal--, 
lines 10 to 11, change "the color-difference signal" to -the chrominance signal--. 
Page 16, 

lines 21 to 22, change "two pieces of chrominance data" to -two kinds of chrominance data 
each by one, two, or four--. 

Page 23, 

line 14, change "it approximates" to —it is approximated--. 

line 19, change "Further," to —In the present invention, however, since degradation in 
resolution of the luminance signal Y in the position where no whole-color-pass filter exists is 
increased in the above-described simple method-. 

Page 27, 

lines 8 to 1 0, delete "in a simple method as follows. Y(h, v)=a X((W(h- 1 , v)+W(h+l, v)+W(h,v- 
l)+W(h,v+l))-4)". 

line 18, change "may be" to -is--. 

IN THE CLAIMS: 

Kindly amend as follows: 

1. (Amended) A solid-state color image pickup apparatus comprising: 
a solid-state image pickup device being provided with; 

a color separation filter having one arrangement pattern comprising four pixels 
adjoining vertically and horizontally, said color separation filter of this arrangement pattern 



comprising two whole-color-pass filters corresponding to two pixel s, a cyan-pass filter corresponding 
to one pixel, and a yellow-pass filter corresponding to one pixel, and repeating the arrangement 
pattern in the vertical and horizontal directions, and 

means for separately taking data corresponding to each pixel of the color separation 

filter, and 

[a signal processing circuit for taking four luminance signals and two kinds of color- 
difference signals, for one of the arrangement patterns, from each pixel data taken out of the solid- 
state image pickup device; at this time, said signal processing circuit generating two of the four 
luminance signals by using only the data from the whole-color-pass filters, and generating the 
remaining two luminance signals by using the data from the whole-color-pass filters and the data of 
peripheral pixels in the vicinity of the four pixels adjoining vertically and horizontally, and generating 
the two kinds of color-difference signals by using the data from the cyan or yellow-pass filter and the 
data from the peripheral pixels ] 

a signal processing circuit for taking four luminance signals and two kinds of color- 
difference signals, for one of the arrangement patterns, from each pixel data taken out of the solid- 
state image pickup device: at this time, said signal processing circuit generating, among the four 
luminance signals, two luminance signals in the positions of the two whole-color- pass filters bv using 
only the data obtained from the positions while generating the remaining luminance signals in the 
positions of the filters transparent to two different kinds of colors bv using the chrominance data 
obtained from the respective positions and the data obtained from peripheral pixels of the four pixels, 
and generating the chrominance signals bv using the date obtained from peripheral pixels of the four 
pixels including the data obtained from the positions of the filters transparent to two different kinds 
of colors. - 

Claim 2, line 7, change "color difference signals" to -chrominance signals-. 



Claim 3, line 7, change "color-difference signals" to -chrominance signals-. 



4. (Amended) A solid-state color image pickup apparatus as defined in Claim 1 further 
comprising: 

storage m eans for capturing a chrominance signal outputted from each pixel 
of the solid-state image pickup device, and storing it: 

correlation calculati on means for calculating the correlation between a target pixel to 
be interpolated and plura l pixels in the vicinity of the target pixel, said target pixel being any of pixels 
in the positions of the filters transparent to two different kinds of colors, which are stored in the 
storage means; and 

interpolation means for interpolating the target pixel in a direction along which the 
calculated correlation is r elatively large, and calculating a luminance signal in the position of the target 
pixel 

[A solid-state color image pickup apparatus as defined in Claim 1: 

wherein said color separation filter having one arrangement pattern comprising 
vertically and horizontally adjoining four pixels, comprises a whole-color-pass filter and a cyan-pass 
filter for the upper two pixels from the left, and a yellow-pass filter and a whole-color-pass filter for 
the lower two pixels from the left; and 

said apparatus further comprising: 

storage means for capturing a chrominance signal outputted from each pixel 
of the solid-state image pickup device, and storing it; 

correlation calculation means for calculating the correlation between a target 
pixel to be interpolated and plural pixels in the vicinity of the target pixel, said target pixel being any 
of a cyan signal pixel and a yellow signal pixel stored in the storage means; and 

interpolation means for interpolating the target pixel in a direction along which 
the calculated correlation is relatively large, and calculating a whole-color-pass signal in the position 
of the target pixel]. 



Claim 13, line 2, change "any of Claims 5 to 10" to -claim 5-. 

line 5, change "color-difference signal" to -chrominance signal-. 



Claim 14, line 2, change any of Claims 5 to 10" to -claim 5- 

line 5, change "color-difference signal" to -chrominance signal--. 

Kindly add the following new claims: 

- 19. A solid-state color image pickup apparatus as defined in claim 6 wherein, the 
correlation calculated by the correlation calculation means is smaller than a given threshold, said 
interpolation means reduces the gain of the chrominance signal corresponding to the pixel. 

20. A solid-state color image pickup apparatus as defined in claim 7 wherein, the 
correlation calculated by the correlation calculation means is smaller than a given threshold, said 
interpolation means reduces the gain of the chrominance signal corresponding to the pixel. 

21. A solid-state color image pickup apparatus as defined in claim 8 wherein, the 
correlation calculated by the correlation calculation means is smaller than a given threshold, said 
interpolation means reduces the gain of the chrominance signal corresponding to the pixel. 

22. A solid-state color image pickup apparatus as defined in claim 9 wherein, the 
correlation calculated by the correlation calculation means is smaller than a given threshold, said 
interpolation means reduces the gain of the chrominance signal corresponding to the pixel. 

23. A solid-state color image pickup apparatus as defined in claim 10 wherein, the 
correlation calculated by the correlation calculation means is smaller than a given threshold, said 
interpolation means reduces the gain of the chrominance signal corresponding to the pixel. 



24. A solid-state color image pickup apparatus as defined in claim 6 wherein, when the 
correlation calculated by the correlation calculation means is smaller than a given threshold, said 
interpolation means reduces the gain of the chrominance signal corresponding to the pixel, stepwise, 
according to the correlation. 

25. A solid-state color image pickup apparatus as defined in claim 7 wherein, when the 
correlation calculated by the correlation calculation means is smaller than a given threshold, said 
interpolation means reduces the gain of the chrominance signal corresponding to the pixel, stepwise, 
according to the correlation. 

26. A solid-state color image pickup apparatus as defined in claim 8 wherein, when the 
correlation calculated by the correlation calculation means is smaller than a given threshold, said 
interpolation means reduces the gain of the chrominance signal corresponding to the pixel, stepwise, 
according to the correlation. 

27. A solid-state color image pickup apparatus as defined in claim 9 wherein, when the 
correlation calculated by the correlation calculation means is smaller than a given threshold, said 
interpolation means reduces the gain of the chrominance signal corresponding to the pixel, stepwise, 
according to the correlation. 

28. A solid-state color image pickup apparatus as defined in claim 1 0 wherein, when the 
correlation calculated by the correlation calculation means is smaller than a given threshold, said 
interpolation means reduces the gain of the chrominance signal corresponding to the pixel, stepwise, 
according to the correlation. ~ 



REMARKS 



The present Preliminary Amendment is submitted to delete the multiple dependency of claims 
13 and 14, thereby placing such claims in condition for examination and reducing the required PTO 
filing fee, and to make preferred changes to the Specification and Claims. 



NEP/kjf 

Washington, D.C. 
Telephone (202) 721-8200 
Facsimile (202) 721-8250 
November 8, 2000 



Respectfully submitted, 




Registration No. 33,145 
Attorney for Applicants 
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DESCRIPTION 
SOLID-STATE COLOR IMAGE PICKUP APPARATUS 



TECHNICAL FIELD 

The present invention relates to a progressive scan type 
solid-state color image pickup apparatus and, more particularly, 
to an arrangement of a color separation filter and a matrix 
calculation method for suppressing degradation in resolution of 
a luminance signal, which occurs when data from the color 
separation filter is subjected to matrix calculation. Further, 
the invention relates to a signal processing method for a solid- 
state color image pickup apparatus which obtains high resolution 
by performing interpolation between pixels. In this method, 
when a color-difference signal is composited from chrominance 
signals outputted from a solid-state image pickup device, the 
frequency characteristic of each chrominance signal is adjusted 
for each color of the color-separation filter to reduce a 
frequency component including aliasing which causes pseudo 
chrominance signals. 

BACKGROUND ART 

A video signal is usually represented by three primary 
colors of light: red (R) , green (G), and blue (B), and it is 
also represented by luminance signals (Y) and two kinds of 



2 



color-difference signals (R-Y, B-Y) . The three primary colors R, 
G, B usually take a form of an input signal to a monitor for a 
computer, and the luminance and color difference signals take a 
form of a digital part in a device such as a TV. Further, in 
recent years, a video signal from a solid-state color image 
pickup apparatus is used not only for image display but also for 
digital recording or video communication between devices. A 
video signal has a great amount of data, and usually it is 
compressed because of the limited recording capacity or 
communication capacity. A format of video signal employed in 
the video compression process is called 4:2:0 or 4:1:1 format in 
which chrominance data is reduced to half as compared with 4:2:2 
format which has conventionally been employed. 

Hereinafter, the conventional solid-state color image 
pickup apparatus will be described with reference to figure 2. 

Figure 2(a) shows the conventional solid-state color image 
pickup apparatus, wherein 1 denotes an optical system for 
forming an image of a subject on the surface of a solid-state 
image pickup device; 2 denotes a solid-state image pickup device 
with a color separation filter for converting the image of the 
subject (optical image) to an image signal (electric signal); 3 
denotes an AD converter for converting the image signal obtained 
by the solid-state image pickup device to a digital image 
signal; and 4 denotes an image signal processing circuit for 
converting the digital image signal to a luminance signal and 
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color-difference signals. Usually, a color separation filter 
having a checkered pattern of complementary colors shown in 
Figure 2(b) comprising magenta (Mg) , green (G), cyan (Cy), and 
yellow (Ye) is used for the color separation filter placed on 
the surface of the solid-state image pickup device 2. 

The operation of the solid-state color image pickup 
apparatus so constituted will be described hereinafter. 

In Figure 2, the subject image (optical image) is formed 
_ on the solid-state image pickup device by the optical system 1. 
The solid-state image pickup device 2 outputs the subject image 
as an image signal which is color-separated by the color 
separation filter. The image signal is converted to a digital 
signal by the AD converter 3, supplied to the signal processing 
circuit 4, and converted to a luminance signal Y and two kinds 
of color-difference signals, whereby a color video signal is 
obtained. The image signal processing circuit generates a 
luminance signal (Y) and a pair of color-difference signals (R-Y, 
B-Y) corresponding to each pixel, from the four pixels of 
complementary colors Mg, G, Cy, and Ye. 

An example of a procedure to generate luminance signals is 
as follows. Luminance signals Y(h,v) are represented by 
Y(0,0) = Mg(0,0)+G(l,0)+Cy(0,l)+Ye(l,l) 
Y(1,0) = G(l,0)+Mg(2,0)+Ye(l,l)+Cy(2,l) 
Y(0,1) = Cy(0,l)+Ye(l,l)+G(0,2)+Mg(l,2) 
Y(l,l) = Ye(l,l)+Cy(2,l)+Mg(l,2)+G(2,2) 



and thus four luminance signals are generated from the outputs 
of the solid-state image pickup device corresponding to the four 
pixels . 

Likewise, color-difference signals R-Y (h,v) are 
represented by 

R-Y(0,0) = Mg(0,0)-G(l,0)-Cy(0,l)+Ye(l,l) 
R-Y(0,1) = -G(l,0)+Mg(2,0)+Ye(l,l)-Cy(2,l) 
R-Y(1,0) = -Cy(0,l)+Ye(l,l)-G(0,2)+Mg(l,2) 
R-Y(l,l) = Ye(l,l)-Cy(2 f l)+Mg(l,2)-G(2,2) 
and color-difference signals B-Y (h,v) are represented by 
B-Y(0,0) = Mg(0,0)-G(l,0)+Cy(0,l)-Ye(l,l) 
B-Y(0,1) = -G(l,0)+Mg<2,0)-Ye<l,l)+Cy(2,l) 
B-Y(l / 0) = Cy(0,l)-Ye(l,l)-G(0,2)+Mg(l,2) 
B-Y(l,l) = -Ye(l,l)+Cy(2,l)+Mg(l,2)-G(2,2) 
The number of the luminance signals Y, color difference 
signals R-Y, and color difference signals B-Y are equal to the 
number of pixels of the solid-state image pickup device, that is, 
these signals take the 4:4:4 format. According to a device to 
which these signals are applied, conversion to 4:2:2, 4:2:0, or 
4:1:1 is carried out. 

Moreover, since pseudo chrominance is generated at edges 
of luminance, edge judgement is performed according to edge 
signals from the composed luminance signals, and the gains of 
the color-difference signals corresponding to pixels decided as 
being edges are reduced to suppress the pseudo chrominance. 



However, in the conventional solid-state color image 
pickup apparatus, no consideration is given to image compression, 
and it is premised that the output signals from the color 
separation filter are in the 4:4:4 format. So, three quarters 
of chrominance data become unnecessary when the output signals 
from the color separation filter are input to a device which is 
predicated on image compression of the 4:2:0 or 4:1:1 format. 
Moreover, each luminance signal is formed from four pixels. For 
example, when Y(0,0) and Y{0,1) shown in Figure 2 (b) are formed, 
since G(1,0) and Ye(l,l) are used for both of them, the 
luminance signals are not simply sampling data and are regarded 
as being passed through a low-pass filter in both of the 
vertical and horizontal directions. Therefore, the resolution 
is degraded as compared with a 3CCD type solid-state image 
pickup device performing pixel by pixel sampling. Also each 
color-difference signal is formed from adjacent four pixels and, 
therefore, it is not simple sampling data but is regarded as 
being passed through a low-pass filter in both of the vertical 
and horizontal directions, whereby the resolution is degraded as 
well. 

In order to solve these problems, in a solid-state color 
image pickup apparatus according to the present invention, a 
color separation filter placed on the surface of a solid-state 
image pickup device has an arrangement pattern of four pixels 
corresponding to two whole-color-pass filters, one cyan-pass 
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filter, and one yellow-pass filter, and this arrangement pattern 
is repeated. The color separation filter so constructed outputs 
four pieces of luminance data, and two kinds of chrominance data 
each by one. Further, when converting them to luminance signals 
and color-difference signals, the relationship between a pixel 
for which an image of its subject is to be formed and its 
peripheral pixels is obtained by correlation detection, and 
pixels existing in the direction of higher correlation are used 
for operation. 

Moreover, in the solid-state image pickup device according 
to the present invention, a plurality of chrominance signals are 
output from a plurality of color-pass filters. These 
chrominance signals are independent from color to color, and 
when attention is directed to a specific chrominance signal, its 
sampling rate is lower than the sampling rate of the whole 
signal. Therefore, there is a possibility that aliasing occurs 
in each chrominance signal, and the chrominance signal includes 
a frequency component having aliasing. 

Figure 15 shows aliasing in the case where a specific 
chrominance signal is interpolated. In Figure 15, the abscissa 
shows the frequency and 2tC indicates the sampling frequency of 
the whole signal, and the ordinate shows the signal amplitude. 
Further, the solid line shows the chrominance signal, and the 
dashed line shows the aliasing. When a color-difference signal 
is composited by interpolation using the chrominance signal 



including a high-frequency component, an aliasing component is 
unfavorably included in the pass band up to K 12 as shown in 
Figure 15. This results in unsatisfactory interpolation 
precision, and causes a pseudo chrominance signal. 

In order to solve the above problem, the solid-state color 
image pickup apparatus according to the present invention is 
provided with a frequency characteristic adjustment means which 
adjusts the frequency characteristic of each chrominance signal 
outputted from a solid-state image pickup device using a color 
separation filter composed of two whole-color-pass filters, one 
cyan-pass filter, and one yellow-pass filter, and a color- 
difference signal is composited by interpolation using the 
chrominance signals having the adjusted frequency characteristic. 

Furthermore, the solid-state color image pickup apparatus 
of the present invention is provided with an edge decision 
function for correlation detection, whereby gains by which the 
color-difference signals are to be multiplied are decided, and 
the corresponding color-difference signals are multiplied by 
these gains, thereby reducing pseudo chrominance. 

Accordingly, the present invention can provide a solid- 
state color image pickup apparatus with reduced deterioration of 
luminance resolution and reduced pseudo chrominance. 

DISCLOSURE OF THE INVENTION 

To solve the above-described problems, a solid-state color 



image pickup apparatus defined in Claim 1 of the present 
invention comprises a solid-state image pickup device and a 
signal processing circuit. The solid-state image pickup device 
is provided with a color separation filter having one 
arrangement pattern comprising four pixels adjoining vertically 
and horizontally, said color separation filter of this 
arrangement pattern comprising two whole-color-pass filters 
corresponding to two pixels, a cyan-pass filter corresponding to 
one pixel, and a yellow-pass filter corresponding to one pixel, 
and repeating the arrangement pattern in the vertical and 
horizontal directions; and means for taking data corresponding 
to each pixel of the color separation filter. The signal 
processing circuit takes four luminance signals and two kinds of 
color-difference signals, for one of the arrangement patterns, 
from each pixel data taken out of the solid-state image pickup 
device and, at this time, the signal processing circuit 
generates two of the four luminance signals by using only the 
data from the whole-color-pass filters, and generates the 
remaining two luminance signals by using the data from the 
whole-color-pass filters and the data from peripheral pixels of 
the four pixels adjoining vertically and horizontally, and 
generates the two kinds of color-difference signals by using the 
data from the cyan or yellow-pass filter and the data from the 
peripheral pixels. Since two of the four luminance signals are 
generated from the data of the whole-color-pass filters, the 



luminance resolution is improved. 

Further, according to the invention defined in Claim 2, in 
the solid-state color image pickup apparatus defined in Claim 1, 
the color separation filter having one arrangement pattern 
comprising vertically and horizontally adjoining four pixels is 
constituted such that the four pixels are vertical two pixels X 
horizontal two pixels, and six signals comprising four luminance 
signals and two kinds of color-difference signals are generated 
from the data taken out of this arrangement pattern, and the six 
signals so generated are output to a device of 4:2:0 format. 
Thereby, the luminance resolution is improved in the device of 
4:2:0 format. 

Further, according to the invention defined in Claim 3, in 
the solid-state color image pickup apparatus defined in Claim 1, 
the color separation filter having one arrangement pattern 
comprising vertically and horizontally adjoining four pixels is 
constituted such that the four pixels are vertical one pixel X 
horizontal four pixels, and six signals comprising four 
luminance signals and two kinds of color-difference signals are 
generated from the data taken out of the arrangement pattern, 
and the six signals so generated are output to a device of 4:1:1 
format. Thereby, the luminance resolution is improved in the 
device of 4:1:1 format. 

Further, according to the invention defined in Claim 4, in 
the solid-state color image pickup apparatus defined in Claim 1, 



the color separation filter having one arrangement pattern 
comprising vertically and horizontally adjoining four pixels, 
comprises a whole-color-pass filter and a cyan-pass filter for 
the upper two pixels from the left, and a yellow-pass filter and 
a whole-color-pass filter for the lower two pixels from the left, 
and the apparatus further comprises storage means for capturing 
a chrominance signal outputted from each pixel of the solid- 
state image pickup device, and storing it; correlation 
calculation means for calculating the correlation between a 
target pixel to be interpolated and plural pixels in the 
vicinity of the target pixel, said target pixel being any of a 
cyan signal pixel and a yellow signal pixel stored in the 
storage means; and interpolation means for interpolating the 
target pixel in a direction along which the calculated 
correlation is relatively large, and calculating a whole-color- 
pass signal in the position of the target pixel. Since an input 
image is converted to luminance signals after performing 
interpolation between pixels of high correlation, degradation of 
luminance resolution is suppressed. 

Further, according to the invention defined in Claim 5, in 
the solid-state color image pickup apparatus defined in Claim 4, 
the correlation calculation means calculates the correlation 
between the target pixel and the pixels in the vicinity of the 
target pixel, including the target pixel, in the horizontal or 
vertical direction. Thereby, degradation of luminance 
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resolution is suppressed in the vertical direction and 
horizontal direction. 

Further, according to the invention defined in Claim 6, in 
the solid-state color image pickup apparatus defined in Claim 4, 
the correlation calculation means calculates the correlation 
between the target pixel and the pixels in the vicinity of the 
target pixel, including the target pixel, in the horizontal or 
vertical direction and, further, in the diagonal direction. 
Thereby, degradation of luminance resolution is suppressed in 
the vertical direction, horizontal direction, and diagonal 
direction. 

Further, according to the invention defined in Claim 7, in 
the solid-state color image pickup apparatus defined in Claim 4, 
the correlation calculation means calculates the correlation 
between the target pixel and the pixels in the vicinity of the 
target pixel, including the target pixel, in the horizontal or 
vertical direction and, further, in the upper right direction, 
or lower right direction, or upper left direction, or lower left 
direction. Thereby, degradation of luminance resolution is 
suppressed in the vertical direction, horizontal direction, 
upper right L-shaped direction, lower right L-shaped direction, 
upper left L-shaped direction, and lower left L-shaped direction. 

Further, according to the invention defined in Claim 8, in 
the solid-state color image pickup apparatus defined in Claim 4, 
the correlation calculation means calculates the correlation 
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between the target pixel and the pixels in the vicinity of the 
target pixel, including the target pixel, in the horizontal or 
vertical direction, and in the diagonal direction, and further, 
in the upper right direction, or lower right direction, or upper 
left direction, or lower left direction. Thereby, degradation 
of luminance resolution is suppressed in the vertical direction, 
horizontal direction,' diagonal direction, upper right L-shaped 
direction, lower right L-shaped direction, upper left L-shaped 
direction, and lower left L-shaped direction. 

Further, according to the invention defined in Claim 9, in 
the solid-state color image pickup apparatus defined in Claim 4, 
the correlation calculation means calculates the correlation by 
performing operation on signals of the same color, between the 
target pixel and the pixels in the vicinity of the target pixel. 
Since the correlation is calculated between signals of the same 
color, the precision of correlation calculation is improved. 

Further, according to the invention defined in Claim 10, 
in the solid-state color image pickup apparatus defined in Claim 
4, the correlation calculation means calculates the correlation 
by performing operation on adjacent pixels being signals of 
different colors, between the target pixel and the pixels in the 
vicinity of the target pixel. Since the correlation is 
calculated between the target pixel and a pixel closest to the 
target pixel, even when these pixels are signals of different 
colors, the precision of correlation calculation is improved. 



Further, according to the invention defined in Claim 11, 
in the solid-state color image pickup apparatus defined in Claim 
4, the interpolation means performs interpolation using only- 
signals of the same color as that of a chrominance signal to be 
generated, in the vicinity of the target pixel, without using 
the chrominance signal of the target pixel in the direction 
where the correlation calculated by the correlation calculation 
means is relatively large. Thereby, the interpolation precision 
is improved, and the luminance resolution is improved. 

Further, according to the invention defined in Claim 12, 
in the solid-state color image pickup apparatus defined in Claim 
4, the interpolation means calculates a shortage of a 
chrominance signal to be generated, from the pixels in the 
vicinity of the target pixel, using the chrominance signal of 
the target pixel in the direction where the correlation 
calculated by the correlation calculation means is relatively 
large, and performs interpolation on the shortage. Since only 
the shortage of chrominance component is interpolated while 
other components are interpolated using the chrominance signal 
component at the target pixel point, the interpolation precision 
is improved and the luminance resolution is prevented from being 
degraded. 

Further, according to the invention defined in Claim 13, 
in the solid-state color- image pickup apparatus defined in any 
of Claims 5 to 10, when the correlation calculated by the 
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correlation calculation means is smaller than a given threshold, 
said interpolation means reduces the gain of the color- 
difference signal corresponding to the pixel. Thereby, a pseudo 
chrominance signal generated at the edge of the luminance signal 
is reduced. 

Further, according to the invention defined in Claim 14, 
in the solid-state color image pickup apparatus defined in any 
of Claims 5 to 10, when the correlation calculated by the 
correlation calculation means is smaller than a given threshold, 
said interpolation means reduces the gain of color-difference 
signal corresponding to the pixel, stepwise, according to the 
correlation. Thereby, a pseudo chrominance signal generated at 
the edge of the luminance signal is adaptively reduced. 

Further, according to the invention defined in Claim 15, 
in the solid-state color image pickup apparatus defined in Claim 
4 , the interpolation means is provided with frequency 
characteristic adjustment means for adjusting the frequency 
characteristic of each chrominance signal outputted from the 
solid-state image pickup device, and the interpolation means 
interpolates and composites a color-difference signal by using 
the chrominance signal whose frequency characteristic is 
adjusted. Thereby, a pseudo chrominance signal, which appears 
when the color-difference signal is composited by interpolation 
using the chrominance signal containing high-frequency 
components, is reduced. 
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Further, according to the invention defined in Claim 16, 
in the solid-state color image pickup apparatus defined in Claim 
5, the interpolation means is provided with frequency 
characteristic adjustment means for adjusting the frequency 
characteristic of each chrominance signal outputted from the 
solid-state image pickup device, and the interpolation means 
interpolates and composites an R-Y color-difference signal in 
the position of the cyan-pass filter, and a B-Y color-difference 
signal in the position of the yellow-pass filter, by using the 
chrominance signal whose frequency characteristic is adjusted. 
Thereby, a pseudo chrominance signal, which appears when the 
color-difference signal is composited by interpolation using the 
chrominance signal containing high-frequency components, is 
reduced. 

Further, according to the invention defined in Claim 17, 
in the solid-state color image pickup apparatus defined in Claim 
15, the interpolation means decides the correlation direction 
according to the correlation calculated by the correlation 
calculation means, performs frequency characteristic adjustment 
when there is a direction where the correlation is relatively 
large, and does not perform frequency characteristic adjustment 
when there is no direction where the correlation is relatively 
large. Thereby, the frequency component is retained when the 
color-difference signal is composited by interpolation using the 
chrominance signal having no correlation direction, whereby the 



reproducibility of the color of image is maintained. 

Further, according to the invention defined in Claim 18, 
in the solid-state color image pickup apparatus defined in Claim 
16, the interpolation means decides the correlation direction 
according to the correlation calculated by the correlation 
calculation means, performs frequency characteristic adjustment 
when there is a direction where the correlation is relatively 
large, and does not perform frequency characteristic adjustment 
when there is no direction where the correlation is relatively 
large. Thereby, the frequency component is retained when the 
color-difference signal is composited by interpolation using the 
chrominance signal having no correlation direction, whereby the 
reproducibility of the color of image is maintained. 

As described above, a solid-state color image pickup 
apparatus according to the present invention is provided with a 
solid-state image pickup device having, at its surface, a color 
separation filter comprising two whole-color-pass filters, one 
cyan-pass filter, and one yellow-pass filter corresponding to 
vertically and horizontally adjoining four pixels, and having a 
repetitive pattern of the four pixels; and a circuit for 
extracting four pieces of luminance data and two pieces of 
chrominance data from the four pixels as the repetitive pattern. 
Thereby, a superior solid-state color image pickup device having 
high luminance resolution and no degradation in the chrominance 
resolution is realized. Further, the above-mentioned apparatus 



is further provided with means for capturing signals outputted 
from the respective pixels; correlation calculation means for 
calculating the correlation between a target pixel to be 
interpolated and plural pixels in the vicinity of the target 
pixel, said target pixel being any of a cyan signal pixel and a 
yellow signal pixel stored in the storage means; and means for 
performing interpolation in a direction along which the 
calculated correlation is relatively large, and calculating a 
whole-color-pass signal in the position of the target pixel. 
Thereby, it is possible to provide a solid-state color image 
pickup device which is able to reduce degradation in the 
luminance resolution, and realize the process of suppressing 
pseudo chrominance signals that appear at edges of the luminance 
signals, without adding a large-scale processing circuit. 

BRIEF DESCRIPTION OF THE DRAWINGS 

Figure 1(a) is a block diagram illustrating a solid-state 
color image pickup apparatus according to the present invention. 

Figure 1(b) is a diagram illustrating a pattern of a color 
separation filter placed on the solid-state image pickup device 
shown in figure 1(a). 

Figure 1(c) is a diagram illustrating a pattern of a color 
separation filter placed on the solid-state image pickup device 
shown in figure 1(a). 

Figure 2(a) is a block diagram illustrating the 
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conventional solid-state color image pickup apparatus. 

Figure 2(b) is a diagram illustrating a pattern of a color 
separation filter placed on the solid-state image pickup device 
shown in figure 2(a). 

Figure 3(a) is a diagram illustrating the positions of 
luminance and color-difference signals in the 4:2:0 format, from 
a solid-state color image pickup apparatus according to a first 
embodiment of the present invention. 

Figure 3(b) is a diagram illustrating the positions of 
luminance and color-difference signals in the 4:1:1 format, from 
a solid-state color image pickup apparatus according to a second 
embodiment of the present invention. 

Figure 4 is a diagram illustrating an arrangement pattern 
of a color separation filter suitable for data output in the 
4:2:0 format, according to the first embodiment of the present 
invention. 

Figure 5 is a diagram illustrating an arrangement pattern 
of a color separation filter suitable for data output in the 
4:1:1 format, according to the second embodiment of the present 
invention . 

Figures 6(a) and 6(b) are diagram for explaining a solid- 
state color image pickup apparatus according to third to seventh 
embodiments of the present invention, wherein figure 6(a) shows 
the structure of the solid-state color image pickup apparatus, 
and figure 6(b) shows the structure of a color-separation filter 
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placed on the solid-state image pickup device. 

Figure 7 is a diagram for explaining calculation of 
correlation, correlation directions (vertical and horizontal 
directions), and interpolation according to a sixth embodiment 
of the present invention. 

Figure 8 is a diagram for explaining calculation of 
correlation, correlation directions (diagonal direction toward 
lower right and diagonal direction toward lower left), and 
interpolation according to a eighth embodiment of the present 
invention . 

Figure 9 is a diagram for explaining calculation of 
correlation, correlation directions (four L-shaped directions), 
and interpolation according to a fifth embodiment of the present 
invention. 

Figure 10 is a diagram for explaining the relationship 
between the correlation and the gain by which the color- 
difference signal is multiplied, according to a ninth embodiment 
of the present invention. 

Figure 11 is a diagram for explaining the relationship 
between the correlation and the gain by which the color- 
difference signal is multiplied, according to the ninth 
embodiment of the present invention. 

Figure 12 is a diagram illustrating the structure of a 
solid-state color image pickup apparatus according to a tenth 
embodiment of the present invention. 



Figure 13 is a diagram for explaining the operation for 
adjusting the frequency characteristic, according to the tenth 
embodiment of the present invention. 

Figure 14 is a diagram for explaining the operation for 
adjusting the frequency characteristic, according to the tenth 
embodiment of the present invention. 

Figure 15 is a diagram for explaining the operation for 
adjusting the frequency characteristic, according to the tenth 
embodiment of the present invention. 

Figure 16 is a diagram illustrating the structure of a 
solid-state color image pickup apparatus according to an 
eleventh embodiment of the present invention. 

BEST MODE TO EXECUTE THE INVENTION 

Hereinafter, embodiments of the present invention will be 
described. 
Embodiment 1. 

Hereinafter, a first embodiment corresponding to Claim 1 
and Claim 2 of the present invention will be described. 

Figure 1(a) illustrates a solid-state color image pickup 
apparatus according to the first embodiment of the invention, 
in the figure, reference numeral 1 denotes an optical system for 
forming an image of a subject on the surface of a solid-state 
image pickup, and it is composed of lenses and the like. 
Reference numeral 2 denotes a solid-state image pickup device 
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with a color separation filter, for converting the subject image 
(optical image) so formed to an image signal (electric signal). 
Reference numeral 3 denotes an AD converter for converting the 
image signal obtained by the solid-state image pickup device to 
a digital image signal. Reference numeral 4 denotes an image 
signal processing circuit for converting the digital image 
signal obtained by the AD converter to luminance signals and 
color-difference signals. 

Figure 1(b) illustrates the color separation filter placed 
on the surface of the solid-state image pickup device 2 shown in 
figure 1(a). This filter has a repetitive pattern comprising 
vertical two pixels X horizontal two pixels . In the color 
separation filter, a whole-color-pass filter and a cyan-pass 
filter are arranged from the left for the upper two pixels, and 
a yellow-pass filter and a whole-color-pass filter are arranged 
from the left for the lower two pixels. 

Figure 3(a) shows input/output signals to/from the image 
signal processing circuit 4 shown in Figure 1(a) according to 
the first embodiment. 

In Figure 1(a), an image of the subject is formed on the 
surface of the solid-state image pickup device through the 
optical system 1, and the subject image (optical image) so 
formed is converted to an image signal (electric signal) by the 
solid-state image pickup device 2 with the color separation 
filter. Then, the image signal obtained from the solid-state 



image pickup device is converted to a digital image signal by 
the AD converter 3, and the digital image signal obtained from 
the AD converter is converted to luminance signals and color- 
difference signals by the image signal processing circuit 4. 
Further, the color separation filter attached to the solid-state 
image pickup device 2 is constructed such that it repeats a 
pattern of vertical two pixels X horizontal two pixels as shown 
in figure 1(b), and the color separation filter having this 
pattern is composed of a whole-color-pass filter and a cyan-pass 
filter for the upper two pixels from the left, and a yellow-pass 
filter and a whole-color-pass filter for the lower two pixels 
from the left, that is, it has two whole-color-pass filters for 
the two pixels among the four pixels, and one cyan-pass filter 
and one yellow-pass filter for the remaining two pixels, 
respectively. In this case, four image signals are obtained 
from the solid-state image pickup device: two pieces of whole 
color data, one piece of cyan data, and one piece of yellow data. 
These four pieces of data are subjected to matrix operation to 
obtain four luminance signals, one R-Y color-difference signal, 
and one B-Y color-difference signal outputted from the image 
signal processing circuit 4. 

Hereinafter, a description will be given of the operation 
of converting the digital image signal to luminance signals and 
color-difference signals by the image signal processing circuit 
4, with reference to figure 3 (a). 



23 



The respective color-pass filters are represented by the 
primary colors of light (red (R), green (G), blue (B) ) as 
follows: W=R+G+G, Cy=G+B, and Ye=R+G. With respect to a 
luminance signal comprising all of the R, G, B components and 
located in a position where the whole-color-pass filter exists, 
this luminance signal is generated from only the signal from the 
whole-color-pass filter which is pure sampling data, and it is 
approximated as follows. 

Y(h,v)=aXW(h,v) 
where a is a factor to adjust the dynamic range, and h+v is 
always an even number in the example of Figure 3(a). 

Further, a luminance signal Y in a position where no 
whole-color-pass filter exists is generated using data of 
peripheral pixels as well, and it approximates as follows in a 
simple method. 

Y(h,v)=ax ( (W(h-l,v)+W(h+l,v)+W(h,v-l)+W(h,v+l) ) -r4) 
where a is a factor to adjust the dynamic range, and h+v is 
always an odd number in the example of Figure 3(a). 

Further, a luminance signal Y in a position where the 
cyan-pass filter exists is generated by interpolation from 
peripheral pixels, utilizing the chrominance data in this 
position which is pure sampling data, since Cy has no R 
component among the luminance signal components, and it is 
approximated as follows . 

R(h,v)=aX (w(h-l,v)+w(h+l,v)+w(h,v-l)+w(h,v+l) -r4 
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-bX (Cy(h,v) X4+Cy<h-2,v)+Cy(h+2,v)+Cy(h,v-2)+Cy(h,v+2) ) -^8 
Y(h,v)=bXCy(h,v)+R(h,v) 

A luminance signal Y in a position where the yellow-pass 
filter exists is generated by interpolation from peripheral 
pixels because Ye has no B component, and it is approximated as 
follows . 

B(h,v)=aX (w(h-l,v)+w(h+l,v)+w(h,v-l)+w(h,v+l) ) t4 

-cX(Ye(h,v) X4+Ye(h-2,v)+Ye(h+2,v)+Ye(h,v-2)+Ye(h,v+2) ) -f 8 
Y(h,v)=cXYe(h,v)+B(h,v) 

where b and c are factors to adjust the dynamic range, h+v is 
always an odd number in the example of Figure 3, h and v in the 
position where the cyan-pass filter exists are an odd number and 
an even number, respectively, and h and v in the position where 
the yellow-pass filter exists are an even number and an odd 
number, respectively. 

The luminance signal Y obtained from the Cy and Ye pixels 
is not regarded as being obtained from pure sampling data by the 
solid-state image pickup device, because the R and B components 
are generated by interpolation using the data of the peripheral 
pixels. However, the G+B component for the Cy pixel and the R+G 
component for the Ye pixel remain as pure sampling data, and the 
R and G components to be interpolated are only one third of the 
luminance signal Y at the maximum and, therefore, these 
components do not adversely affect the luminance signal Y, 
resulting in the luminance signal Y maintaining high resolution. 



Furthermore, with respect to the color-difference signals 
{R-Y, B-Y), one piece of data is taken out for each color- 
difference signal for four luminance signals, and it is 
converted to R, G, B using, as a pattern, vertical two pixels X 
horizontal two pixels corresponding to the luminance signals, in 
a simple calculation method. The R and B components for 
conversion to the color-difference signals are represented as 
follows . 

R(h,v)=aXW{ (h div 2)*2,(v div 2)*2)-bXCy{ (h div 2)*2+l, 
(v div 2)*2) 

B(h,v)=aXW( (h div 2)*2+l,(v div 2)*2+l)-cXYe{ (h div 2>*2, 
(v div 2)*2+l) 
Further, the G component is represented as follows. 

G(h,v)=aX (W( (h div 2)*2,(v div 2)*2)+W((h div 2)*2+l, 
(v div 2)*2+l ) ) -=-2-R(h,v)-B(h,v) 
According to these R, G, B, the color-difference signals are 
approximated as follows . 

R-Y(h,v)=2 XR(h,v)-G(h,v) 
B-Y(h,v)=2 XB(h,v)-G(h,v) 
where a, b, and c are factors to adjust the dynamic range, div 
means a calculation which takes only the quotient obtained by 
division of integers and omits the remainder, and * means 
multiplication . 

A pair of color-difference signals so obtained have 
improved chrominance resolution because the output of the solid- 



state image pickup device is not used for adjacent other color- 
difference signals. 

Through the above-mentioned operation, as shown in figure 
3(a), two kinds of color-difference signals R-Y and B-Y are 
obtained, each by one, for four luminance signals Y 
corresponding to vertical two pixels X horizontal two pixels, 
and these signals are suitable as input signals to a device of 
the 4:2:0 format . 

in addition, when the transmissivity of the whole-color- 
pass filter is set at 0.3 for R, 0.59 for B, and 0.11 for G, a 
pure luminance signal is obtained because the ratio of the 
transmissivities is equal to the mixing ratio of primary colors 
of the luminance signal Y, resulting in further improved 
resolution. That is, usually, the respective color-pass filters 
are expressed as follows with the primary color components (red, 
green, blue (R, G, B) ) : W=R+G+B, Cy=G+B, Ye=R+G. However, when 
the above-described setting is carried out, the RGB mixing ratio 
of the luminance signal Y is represented by 

Y=0 . 30R+0 . 59G+0 . 11B 
and the transmissivities of the respective color-pass filters 
used for this first embodiment are adjusted as follows. 

W=0 . 30R+0 . 59G+0 . 11B 

Cy=(0.59G+0.1lB) -fO.7 

Ye=(0.30R+0.5 9G) -i-0.89 
At this time, a luminance signal Y in a position where the 
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whole-color-pass filter exists is generated from the signal 
outputted from the whole-color-pass filter, which becomes pure 
sampling data, and this luminance signal is represented by 
Y(h,v)=aXw(h,v) 

where a is a factor for adjusting the dynamic range, and h+v is 
always an even number in the example of Figure 3 (a). Further, 
a luminance signal Y in a position where no whole-color-pass 
filter exists is generated using data of peripheral pixels, in a 
simple method as follows. 

Y(h,v)=aX ( (W(h-l,v)+W(h+l,v)+W(h,v-l)+W(h,v+l) ) -M) 
Further, utilizing the chrominance data in this position which 
is pure sampling data, a luminance signal Y in a position where 
the cyan-pass filter exists is generated as follows. 

Y(h,v)=aX ( (W(h-l,v)+W(h+l,v)+W(h,v-l)+W(h,v+l) ) ^4) 
+0. 7 XbX (Cy(h-2,v) 2-(Cy (h+2,v)+Cy (h,v-2 ) 
+Cy(h,v+2)+Cy(h,v) )t8) 

Further, a luminance signal Y in a position where the 
yellow-pass filter exists may be generated as follows. 

Y(h,v)=aX ( (W{h-l,v)+W(h+l,v)+W(h,v-l)+W(h,v+l) ) -r4) 
+0.7X C X (Ye(h-2,v)-^2-(Ye(h+2,v)+Ye(h,v-2) 
+Ye ( h , v+2 ) +Ye ( h , v ) ) 4- 8 ) 
where b and c are factors for adjusting the dynamic range, and 
h+v is always an odd number in the example of Figure 3 (a), h 
and v in the position where the cyan-pass filter exists are an 
odd number and an even number, respectively, and h and v in the 
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position where the yellow-pass filter exists are an even number 
and an odd number, respectively. 

Moreover, with respect to the color-difference signals (R- 
Y, B-Y), one piece of data is taken out for each color- 
difference signal for four luminance signals, and the color- 
difference signals are obtained using, as a pattern, vertical 
two pixels X horizontal two pixels corresponding to the 
luminance signals, in a relatively simple calculation method as 
follows . 

R-Y(h,v) = 0.7T0.3X(aX(W((h div 2)*2,(v div 2)*2) 

+W((h div 2)*2+l,(v div 2)*2+l) ) 42-bXCy( (h div 2)*2+l, 
(v div 2 )*2 ) ) 

B-Y(h,v)=0.8940.11X (aX (W( (h div 2)*2,(v div 2)*2) 

+W((h div 2)*2+l,(v div 2 )*2+l ) ) 42-bX Ye( (h div 2)*2, 
(v div 2 )*2 + l ) ) 

Alternatively, considering the sampling positions of the color- 
difference signals, the color difference signals may be obtained 
as follows. 

R-Y(h,v)=0.740.3X (aX (W( (h div 2)*2,<v div 2)*2) 

+W((h div 2)*2+l,(v div 2 )*2+l ) ) 42-bX (Cy( (h div 2)*2+l, 
(v div 2)*2) X2+Cy( (h div 2)*2-l,(v div 2)*2) 
+Cy((h div 2)*2+l,(v div 2)*2+2))44) 

B-Y{h,v)=0.8940.11X (aX (W( (h div 2)*2,(v div 2)*2) 

+W((h div 2)*2+l,(v div 2)*2+l) ) 42-bX (Ye( (h div 2)*2, 
(v div 2)*2+l)X2 Ye((h div 2)*2,(v div 2)*2-l) 
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+Ye((h div 2)*2+2,(v div 2)*2+l))-r4 
in this first embodiment, the whole-color-pass filters are 
arranged checkerwise. However, as shown in figure 4(a), in a 
repetitive pattern comprising vertical two pixels X horizontal 
two pixels, two whole-color-pass filters may be adjacent to each 
other in the horizontal direction, by arranging the two whole- 
color-pass filters for the upper two pixels, and a cyan-pass 
filter and a yellow-pass filter for the lower two pixels. in 
this case, the horizontal resolution is improved. Further, as 
shown in figure 4(b), two whole-color-pass filters may be 
adjacent to each other in the vertical direction, by arranging a 
whole-color-pass filter and a cyan-pass filter for the upper two 
pixels, and a whole-color-pass filter and a yellow-pass filter 
for the lower two pixels. in this case, the vertical resolution 
is improved. Further, the arrangement of the filters 
corresponding to the four pixels may be varied from pattern to 
pattern with the same effect as mentioned above. 

Further, while in this first embodiment the color 
separation filter outputs signals in the 4:2:0 format, it can 
output signals in the 4:4:4, 4:2:2, or 4:4:1 format because the 
R, G, B components can be placed in any position by 
appropriately utilizing the peripheral pixels, although the 
chrominance resolution is somewhat degraded when performing 
conversion to the color-difference signals. 

Further, while in this first embodiment the cyan-pass 



filter and the yellow-pass filter are employed as filters other 
than the whole-color-pass filter, these filters may be a red- 
pass filter and a blue-pass filter. In this case, the 
calculation is simplified because there is no necessity of 
taking the R and B components from the filters. However, since 
the red-pass filter and the blue-pass filter include no green 
components, green components should be supplied from the 
peripheral pixels, whereby the luminance resolution is degraded. 

Furthermore, in this first embodiment, the color 
separation filter having a pattern of vertical two pixels X 
horizontal two pixels is composed of a whole-color-pass filter 
and a cyan-pass filter for the upper two pixels from the left, 
and a yellow-pass filter and a whole-color-pass filter for the 
lower two pixels from the left. However, this arrangement may 
be changed without changing the colors (i.e., cyan and yellow) 
of the color-pass filters, or the combination of the colors of 
the color-pass filters may be changed from cyan and yellow to 
cyan and magenta, magenta and yellow, red and blue, red and 
green, or green and blue. 
Embodiment 2 . 

Hereinafter, a second embodiment corresponding to Claim 1 
and Claim 3 of the present invention will be described. 

Figure 1(c) shows a color separation filter attached to 
the surface of the solid-state image pickup device 2 shown in 
figure 1(a). This filter has a repetitive pattern of vertical 



one pixel X horizontal four pixels, and comprises a whole- 
color-pass filter, a cyan-pass filter, a whole-color-pass filter, 
and a yellow-pass filter which are arranged in this order from 
the left of the pattern. 

Figure 3(b) shows input/output signals to/from the image 
signal processing circuit 4 shown in figure 1(a). 

In figure 1(a), an image of a subject is formed at the 
surface of the solid-state image pickup device through the 
optical system 1, and the subject image (optical image) so 
formed is converted to an image signal (electric signal) by the 
solid-state image pickup device 2 with the color separation 
filter. Then, the image signal obtained from the solid-state 
image pickup device is converted to a digital image signal by 
the AD converter 3, and the digital image signal obtained from 
the AD converter is converted to luminance signals and color- 
difference signals by the image signal processing circuit 4. 
Further, as shown in figure 1(c), the color separation filter 
attached to the solid-state image pickup device 2 has a 
repetitive pattern of vertical one pixel X horizontal four 
pixels, and comprises, from the left, a whole-color-pass filter, 
a cyan-pass filter, a whole-color-pass filter, and a yellow-pass 
filter. That is, the color separation filter has two whole- 
color-pass filters for two pixels among four pixels, and a cyan- 
pass filter and a yellow-pass filter for the remaining two 
pixels. In this case, an image signal obtained from the solid- 
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state image pickup device comprises four pieces of data, i.e., 
two pieces of whole color data, one piece of cyan data, and one 
piece of yellow data. The image signal processing circuit 4 
subjects these four pieces of data to matrix calculation, and 
outputs four luminance signals, one R-Y color-difference signal, 
and one B-Y color difference signal. 

Hereinafter, a description will be given of the operation 
for converting the digital image signal to luminance signals and 
color-difference signals by the image signal processing circuit, 
with reference to figure 3(b). 

The respective color-pass filters are represented by the 
primary color components of light (red (R), green (G), blue (B)) 
as follows: W=R+G+B, Cy=G+B, and Ye=R+G. A luminance signal Y 
in a position where the whole-color-pass filter exists, whose 
luminance data contains all of the R, G, B components, is 
generated from a signal outputted from the whole-color-pass 
filter which becomes pure sampling data, and it is approximated 
as follows . 

Y(h,v)=aXW(h,v) 

where a is a factor to adjust the dynamic range, and h is always 
an even number in the example of Figure 3 (b) . 

Further, a luminance signal in a position where no whole- 
color-pass filter exists is generated using data of peripheral 
pixels, and it is approximated as follows in a simple generation 
method. 
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Y(h,v)=aX( (W(h-1, v)+W(h+l,V) ) 4-2) 
where a is a factor to adjust the dynamic range, and h is always 
an odd number in the example of Figure 3 (b). 

Moreover, effectively utilizing the chrominance data as 
pure sampling data in the corresponding position, a luminance 
signal Y in a position where the cyan-pass filter exists is 
generated by interpolation from peripheral pixels because the Cy 
has no R components, and it is approximated as follows. 
R(h,v)=aX ( (w(h-l,v)+w(h+l,v) ) X2+w(h-l,v-l)+w(h+l,v-l) 

+w(h-l,v+l)+w(h+l,v+l))T8-bX (Cy(h,v) X2+Cy(h,v-1) 
+Cy (h,v+l ) ) 4-4 
Y(h,v)=bXCy(h,v)+R(h,v) 

Further, a luminance signal Y in a position where the 
yellow-pass filter exists is generated by interpolation from 
peripheral pixels because the Ye has no B components, and it is 
approximated as follows . 

B(h,v)=aX ( (w(h-l,v)+w(h+l,v) ) X2+w<h-l,v-l)+w(h+l,v-l) 

+w(h-l,v+l)+w(h+l,v+l) )t8-cX (Ye(h,v) X2+Ye(h,v-1) 
+Ye(h,v+1) 4-4 

Y(h,v)=cXYe(h,v)+B(h,v) 

where b and c are factors to adjust the dynamic range, and mod 
means a calculation to take only the remainder obtained by 
division of integers, whereby h in the position where the cyan- 
pass filter exists becomes (h mod 4)=1, and h in the position 
where the yellow-pass filter exists becomes (h mod 4) =3. 
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The R and B components required for obtaining the above- 
described luminance signal Y from the Cy and Ye pixels are 
generated by interpolation using the data of the peripheral 
pixels and, therefore, the luminance signal Y is not regarded as 
being obtained from pure sampling data by the solid-state image 
pickup device. However, the G+B components for the Cy pixels 
and the R+G components for the Ye pixels remain as pure sampling 
data, and the R and G components to be interpolated are one 
third of the luminance signal Y at the maximum and do not 
adversary affect the luminance signal Y, resulting in the 
luminance signal Y maintaining high resolution. 

With respect to the color-difference signals (R-Y, B-Y), 
one piece of data is taken out for each color-difference signal 
for four luminance signals and, in a simple calculation method, 
it is converted to R, G, B using, as a pattern, vertical one 
pixel x horizontal four pixels corresponding to the luminance 
signals. The R and B components for this conversion are 
represented by 

R(h,v)=aXW( (h div 4)*4,v)-bXCy( (h div 4)*4+l,v) 

B(h,v)=aXW( (h div 4 ) *4+2 ,v) -c X Ye ( (h div 4)*4+3,v) 
and the G component is represented by 
G(h,v)=aXW( (h div 4)*4,v)+W((h div 4)*4+2,v) -r2-R{h,v)-B(h,v) 

Using these R, G, B components, the color-difference 
signals are approximated as follows. 

R-Y(h,v)=2 XR(h,v)-G(h,v) 
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B-Y(h,v)=2XB(h,v)-G(h,v) 
where a, b and c are factors to adjust the dynamic range. 

A pair of color-difference signals so obtained have 
improved color resolution because the output of the solid-state 
image pickup device is not used for adjacent other color- 
difference signals. 

Through the above-mentioned operation, as shown in figure 
3(b), two kinds of color-difference signals R-Y and B-Y are 
obtained, each by one, for the four luminance signals Y 
corresponding to vertical one pixel X horizontal four pixels, 
and these signals are suitable as input signals to a device 
employing the 4:1:1 format. 

In addition, when the transmissivity of the whole-color- 
pass filter is set at 0.3 for R, 0.59 for B, and 0.11 for G, a 
pure luminance signal is obtained because the ratio of the 
transmissivities is equal to the mixing ratio of primary colors 
of the luminance signal Y, resulting in further improved 
resolution. The transmissivities can be obtained by a 
calculation method similar to that described for the first 
embodiment although the matrix is different from that of the 
first embodiment . 

Further, as described for the first embodiment, not only 
4:1:1 output but also 4:4:4, 4:2:2, or 4:2:0 output can be 
achieved because the R, G, B components can be placed in any 
position by appropriately utilizing the peripheral pixels 



although the chrominance resolution is somewhat degraded when 
performing conversion to the difference-color signals. 

As shown in figure 5(a), the color-pass filters may be 
shifted by one pixel when the pattern of vertical one pixel x 
horizontal four pixels shown in figure 1(c) or 3(b) is repeated 
in the vertical direction so as to arrange the whole-color-pass 
filters checkerwise. In this case, the resolution of the 
luminance signal in the diagonal direction can be improved. 
Further, as shown in figure 5(b), the whole-color-pass filters 
and the color-pass filters other than the whole-color pass 
filters shown in figure 1(c) may be interchanged and, further, 
the cyan-pass filter and the yellow-pass filter may be 
interchanged, providing a repetitive pattern of 8 pixels. This 
filter arrangement provides improved chrominance resolution and 
is suitable for both of the 4:1:1 format (vertical one pixel X 
horizontal four pixels) and the 4:2:0 format (vertical two 
pixels X horizontal two pixels ) . 

In this second embodiment, the color separation filter 
having a pattern of vertical one pixel X horizontal four pixels 
is composed of, from the left, a whole-color-pass filter, a 
cyan-pass filter, a whole-color-pass filter, and a yellow-pass 
filter. However, the arrangement of these filters may be 
changed without changing the colors (i.e., cyan and yellows) of 
the color-pass filters, or the combination of the colors of the 
color-pass filters may be changed from cyan and yellow to cyan 
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and magenta, magenta and yellow, red and blue, red and green, or 
green and blue. In the repetitive pattern of the color-pass 
filters corresponding to four pixels, when providing two whole- 
color-pass filters and two kinds of color-pass filters 
transparent to colors other than the whole color, the same 
effects as mentioned above are achieved. Further, four patterns 
of the color separation filter, each comprising vertical one 
pixel X horizontal four pixels, are provided in the vertical 
direction, and the arrangements of the color-pass filters in 
these four patterns may be different from each other. 
Embodiment 3 . 

Hereinafter, a third embodiment corresponding to Claims 4, 
5, 9, and 12 of the present invention will be described with 
reference to figures 6 and 7 . 

In figure 6(a), reference numeral 1 denotes an optical 
system for making an image of a subject on the surface of a 
solid-state image pickup device, and it is composed of lenses 
and the like. Reference numeral 2 denotes a solid-state image 
pickup device with a color separation filter, for converting the 
subject image (optical image) so formed to an image signal 
(electric signal). Reference numeral 3 denotes an AD converter 
for converting the image signal obtained from the solid-state 
image pickup device 2 to a digital image signal. Reference 
numeral 5 denotes a storage circuit for storing one frame of the 
digital image signal obtained in the AD converter 3. Reference 
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numeral 6 denotes a correlation calculation circuit for 
calculating the correlation between an arbitrary pixel of the 
digital image signal stored in the storage circuit 5, and 
peripheral pixels. Reference numeral 7 denotes an interpolation 
circuit for performing interpolation according to the 
correlation calculated by the correlation calculation circuit 6 
to output luminance signals and color-difference signals. The 
luminance signals and the color-difference signals are generated 
by the optical system 1 , the solid-state image pickup device 2 
with the color separation filter, the AD converter 3, the 
storage circuit 5, the correlation calculation circuit 6, and 
the interpolation circuit 7. 

Figure 6(b) shows the structure of the color separation 
filter placed on the solid-state image pickup device 2. This 
color separation filter has, as one unit of arrangement, four 
pixels adjacent to each other in the vertical and horizontal 
directions, and this arrangement unit is composed of a whole- 
color-pass filter and a cyan-pass filter for the upper two 
pixels from the left, and a yellow-pass filter and a whole- 
color-pass filter for the lower two pixels from the left. This 
arrangement unit is repeated in the vertical and horizontal 
directions . 

The w pixel, Cy pixel, and Ye pixel captured in a storage 
circuit (not shown) are expressed by R, G, B components as 
follows: W=(R+G+B)/3, Cy=(G+B)/3, and Ye=(R+B)/3. When W = Y, 
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the output signal corresponding to the W pixel can be expressed 
as a luminance signal. With respect to the Cy pixel and Ye 
pixel, the R and B components are obtained by interpolation and 
added to the Cy pixel and Ye pixel, respectively, whereby a 
luminance signal is expressed. Although signals corresponding 
to peripheral pixels are used for interpolation, the peripheral 
pixels used for interpolation are decided by calculating the 
correlation with a target pixel to be interpolated, by the 
correlation calculation circuit 6. Initially, the correlation 
calculating method will be described. 

Figure 7 shows the arrangement of peripheral pixels when a 
cyan pixel Cyn is a target pixel. in figure 7, # and O 
indicate Ye pixels and W pixels which are not used in the 
interpolation process for the pixel Cyn. Assuming that the 
correlation in the vertical direction ((D-CD 1 direction in figure 
7) is Vc and the correlation in the horizontal direction 
direction in figure 7) is He, these correlations Vc and He are 
calculated using the following expressions. 

Vc= | Wu-Wd | + 1 Cyu-Cyn | + 1 Cyd-Cyn | ( 1 ) 

Hc= | Wl-Wr | + | Cyl-Cyn | + | Cyr-Cyn | ( 2 ) 

Using the result of the calculation, the correlation 
direction is decided according to the following conditional 
expressions . 

Vc+Th < He (3) 
Hc+Th < Vc (4) 



wherein Th is a threshold, and this is a specific constant. The 
correlation direction is decided as the vertical direction when 
the expression (3) holds and as the horizontal direction when 
the expression (4) holds. When none of the expressions (3) and 
(4) holds, it is decided that there is no correction direction. 

Next, the interpolation process will be described. 

When the correlation direction is the vertical direction, 
a shortage component RCy is calculated using, as pixels for 
interpolation, only the peripheral pixels in the vertical 
direction with respect to the target pixel Cyn, according to the 
following expression. 

RCy= ( Wu+Wd ) 12 - ( 2*Cy n+Cyu+Cyd ) / 4 ( 5 ) 

When the correlation direction is the horizontal direction, 
a shortage component RCy is calculated using only the peripheral 
pixels in the horizontal direction with respect to the target 
pixel Cyn, according to the following expression. 

RCy=(Wl+Wr ) /2- ( 2*Cyn+Cyl+Cyr ) /4 ( 6 ) 

When there is no correlation direction, a shortage 
component RCy is calculated using the peripheral pixels in both 
of the horizontal and vertical directions with respect to the 
target pixel Cyn, according to the following expression. 

RCy=(Wu+Wd+Wl+Wr ) /4- ( 4*Cyn+Cyu+Cyd+Cyl+Cyr ) /8 ( 7 ) 

Using the shortage component RCy obtained by any of the 
expressions (5) to (7), the W component of the target pixel Cyn 
is obtained as follows. 
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W * =Cyn+RCy 

For all of target pixels Cyn, w is calculated by the 
above-described operation. 

when the target pixel is a Ye pixel, Cy in the expressions 
(1) and (2) is replaced with Ye to calculate the correlation, 
and Cy in the right sides of the expressions (5) to (7) is 
replaced with Ye to obtain a shortage component BYe. Since the 
shortage component so obtained is not the R component but the B 
component, the W component of the Ye pixel can be obtained by 
W=Yen+Bye. For all of the target pixels Yen, w is calculated 
by the above-described operation. 

In this interpolation process, the luminance W of the Cy 
pixel and the luminance W of the Ye pixel are obtained, whereby 
all of the luminance signals are obtained. In this method, the 
signals of the w pixels are used as they are, and the Cy pixels 
and Ye pixels are interpolated using the peripheral pixels 
having the higher correlation with them, whereby degradation in 
resolution is suppressed. 

As described above, according to the third embodiment, the 
correlation between the target pixel to be interpolated and the 
peripheral pixels in the vertical and horizontal directions 
including the target pixel is detected, and the target pixel is 
interpolated using the correlation. Therefore, highly precise 
luminance signals are obtained, and the resolution is prevented 
from being degraded. 



I 
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Embodiment 4 . 

Hereinafter, a fourth embodiment corresponding to Claim 6 
of the present invention will be described with reference to 
figure 8. 

The construction of this fourth embodiment is identical to 
that of the third embodiment. In this fourth embodiment, the 
correlation calculation circuit 6 calculates the correlation in 
the diagonal direction as well, and the interpolation circuit 7 
performs interpolation using the correlation in the diagonal 
direction. 

Hereinafter, the correlation calculating method will be 
described. 

Figure 8 shows the arrangement of peripheral pixels when a 
cyan pixel Cyn is a target pixel to be interpolated. In figure 
8, • and O indicate Ye pixels and w pixels which are not used 
in the interpolation process for the pixel Cyn. 

In the third embodiment of the invention, only the 
correlations in the (D-CD 1 direction and the d)-© 1 direction are 
obtained as shown in figure 7. In this fourth embodiment, 
assuming that the correlation in the diagonal direction toward 
the lower right (the direction in figure 8) is Nr, and the 

correlation in the diagonal direction toward the lower left (the 
<D-@' direction in figure 8) is Nl, and these correlations Nr 
and Nl are calculated using the following expressions. 

Nr=| (Wu+Wl)/2-(Wd+Wr)/2|+| (Cyul-Cyn) |+| (Cydr-Cyn) | (8) 
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Nl=| (Wu+Wr)/2-(Wd+Wl)/2|+j (Cydl-Cyn) |+| (Cyur-Cyn) j (9) 
Using the result of this calculation and the Vc and He 
obtained from the expressions (1) and (2), the correlation 
direction is decided according to the following conditional 
expressions . 

Vc+Th < min(Hc,Nr,Nl) (10) 

Hc+Th < min(Vc,Nr,Nl) (11) 

Nr+Th < min(Hc,Vc,Nl) (12) 

Nl+Th < min(Hc,Vc,Nr) (13) 
wherein Th is a threshold (a specific constant), and min is a 
function which takes the minimum value among the elements in the 
parentheses. The correlation direction is the vertical 
direction when the expression (10) holds, the horizontal 
direction when the expression (11) holds, the diagonal direction 
toward the lower right when the expression (12) holds, and the 
diagonal direction toward the lower left when the expression 
(13) holds. When none of the expressions (10) to (13) holds, 
there is no correlation direction. 

Next, the interpolation process will be described. 
While in the third embodiment the interpolation process is 
described for the case where the correlation direction is 
decided as the vertical or horizontal direction and the case 
where there is no correlation direction, in this fourth 
embodiment the interpolation process in the case where the 
correlation direction is decided as the diagonal direction will 
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be added. 

When the correlation direction is the diagonal direction 
toward the lower right, a shortage component RCy is calculated 
using only the peripheral pixels in the diagonal direction 
toward the lower right with respect to the target pixel Cyn, 
according to the following expression. 

RCy= ( Wu+Wd+Wl+Wr ) / 4 - ( 2 *Cyn+Cyul+Cydr ) / 4 (14) 
When the correlation direction is the diagonal direction 
toward the lower left, a shortage component RCy is calculated 
using only the peripheral pixels in the diagonal direction 
toward the lower left with respect to the target pixel Cyn, 
according to the following expression. 

RCy= ( Wu+Wd+Wl+Wr ) /4- ( 2*Cyn+Cyur+Cydl ) /4 (15) 
Thereafter, interpolation is performed in like manner as 
described for the third embodiment, thereby all of the luminance 
signals are obtained. 

As described above, in this fourth embodiment, since 
interpolation is performed using not only the correlations in 
the vertical and horizontal directions but also the correlations 
in the diagonal directions, degradation of resolution is 
suppressed in the diagonal directions as well as in the vertical 
and horizontal directions. 
Embodiment 5. 

Next, a fifth embodiment corresponding to Claim 7 of the 
present invention will be described with reference to figure 9. 



The construction of this fifth embodiment is identical to 
that of the third embodiment. In this fifth embodiment, the 
correlation calculation circuit 6 calculates the correlations in 
L-shaped directions as well, and the interpolation circuit 7 
performs interpolation using the correlations in the L-shaped 
directions . 

Initially, the correlation calculation method will be 
described. 

Figure 9 shows the arrangement of peripheral pixels when a 
cyan pixel Cyn is a target pixel. In figure 9, # and O 
indicate Ye pixels and W pixels which are not used in the 
interpolation process for the pixel Cyn. 

In the third embodiment of the invention, only the 
correlations in the (D-d)' direction and the @-(D' direction are 
obtained as shown in figure 7. In this fifth embodiment, 
assuming that the correlation in the upper left L-shaped 
direction (the ©-(§)' direction in figure 9) is Lul, the 
correlation in the upper right L-shaped direction (the (§)-©' 
direction) is Lur, the correlation in the lower left L-shaped 
direction (the ©-© ' direction) is Ldl, and the correlation in 
the lower right L-shaped direction (the (D-® ' direction) is Ldr, 
and these correlations are calculated using the following 
expressions . 

Lul= | Wu-Wl | + | Cyu-Cyn | + | Cyl-Cyn | (16) 
Lur= | Wu-Wr | + | Cyu-Cyn [ + j Cyr-Cyn | (17) 



Ldl= j Wd-Wl | + 1 Cyd-Cyn | + 1 Cyl-Cyn | ( 18 ) 

Ldr= | Wd-Wr | + | Cyd-Cyn | + | Cyr-Cyn | (19) 
Using the result of calculation and the Vc and He obtained 
from the expressions (1) and (2), the correlation direction is 
decided according to the following conditional expressions. 
Vc+Th < min(Hc,Lul,Ldl,Lur,Ldr) (20) 
Hc+Th < min(Vc,Lul,Ldl,Lur,Ldr) (21) 
Lul+Th < min(Hc,Vc,Ldl,Lur,Ldr) (22) 
Lur+Th < min(Hc,Vc,Lul,Ldl,Ldr) (23) 
Ldl+Th < min(Hc,Vc,Lul,Lur,Ldr ) (24) 
Ldr+Th < min(Hc,Vc,Lul,Ldl,Lur) (25) 
where Th is a threshold (a specific constant), and min is a 
function which takes the minimum value among the elements in the 
parentheses. The correlation direction is the vertical 
direction when the expression (20) holds, the horizontal 
direction when the expression (21) holds, the upper left L- 
shaped direction when the expression (22) holds, the upper right 
L-shaped direction when the expression (23) holds, the lower 
left L-shaped direction when the expression (24) holds, and the 
lower right L-shaped direction when the expression (25) holds. 
When none of the expressions (20) to (25) holds, there is no 
correlation direction. 

Next, the interpolation process will be described. 
While in the third embodiment the interpolation process is 
described for the case where the correlation direction is the 
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vertical or horizontal direction and the case where there is no 
correlation direction, in this fifth embodiment the 
interpolation process in the case where the correlation 
direction is any of the above-mentioned L-shaped directions will 
be added. 

When the correlation direction is the upper left L-shaped 
direction, a shortage component RCy is calculated using only the 
peripheral pixels in the upper left L-shaped direction with 
respect to the target pixel Cyn, according to the following 
expression. 

RCy= ( Wu+Wl ) / 2 - ( 2 *Cyn+Cyu+Cy 1 ) / 4 (26) 
When the correlation direction is the upper right L-shaped 
direction, a shortage component RCy is calculated using only the 
peripheral pixels in the upper right L-shaped direction with 
respect to the target pixel Cyn, according to the following 
expression. 

RCy= ( Wu+Wr ) / 2 - ( 2 *Cy n+Cyu+Cy r ) / 4 (27) 
When the correlation direction is the lower left L-shaped 
direction, a shortage component RCy is calculated using only the 
peripheral pixels in the lower left L-shaped direction with 
respect to the target pixel Cyn, according to the following 
expression. 

RCy= ( Wd+Wl ) / 2 - ( 2*Cyn+Cyd+Cyl ) / 4 (28) 
When the correlation direction is the lower right L-shaped 
direction, a shortage component RCy is calculated using only the 
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peripheral pixels in the lower right L-shaped direction with 
respect to the target pixel Cyn, according to the following 
expression. 

RCy = ( Wd+Wr ) / 2 - ( 2 *Cyn+Cyd+Cyr ) / 4 (29) 
Thereafter, interpolation is carried out in like manner as 
described for the third embodiment, whereby all of the luminance 
signals are obtained. 

According to the fifth embodiment, as described above, not 
only the correlations in the vertical and horizontal directions 
but also the correlations in the respective L-shaped directions 
are detected to be used for interpolation. Therefore, 
degradation in resolution is suppressed in the L-shaped 
directions as well as in the vertical and horizontal directions. 
Embodiment 6. 

Next, a sixth embodiment corresponding to Claim 8 of the 
present invention will be described. 

The construction of this sixth embodiment is fundamentally 
identical to that of the third embodiment. in this sixth 
embodiment, however, the correlation calculation circuit 6 
calculates the correlations in the diagonal directions and the 
L-shaped directions as well as correlations in the vertical and 
horizontal directions, and the interpolation circuit 7 performs 
interpolation using the correlations in the diagonal directions 
and the L-shaped directions as well. 

Initially, the correlation calculation method will be 
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described. 

In the third embodiment, only the correlations Vc and He 
in the ®-®' direction and the (D-©' direction are obtained as 
shown in figure 7. In this sixth embodiment, more correlations 
as follows are obtained in like manner as described for the 
fourth and fifth embodiments: the correlation Nr in the diagonal 
direction toward the lower right (the (S>-®' direction in figure 
8), the correlation Nl in the diagonal direction toward the 
lower left (the @-(D' direction), the correlation Lul in the 
upper left L-shaped direction (the d)-(D' direction in figure 9), 
the correlation Lur in the upper right L-shaped direction (the 
(§)-©' direction), the correlation Ldl in the lower left L-shaped 
direction (the <2)-(7)' direction), and the correlation Ldr in the 
lower right L-shaped direction (the (H)-®' direction). 

Using the results, the correlation direction is decided by 
the following conditional expressions. 

Vc+Th < min(Hc,Nr,Nl,Lul,Lur,Ldl,Ldr) (30) 
Hc+Th < min(Vc,Nr,Nl,Lul,Lur,Ldl,Ldr) (31) 
Nr+Th < min(Hc,Vc,Nl,Lul,Lur,Ldl,Ldr) (32) 
Nl+Th < min(Hc,Vc,Nr,Lul,Lur,Ldl,Ldr) (33) 
Lul+Th < min(Hc,Vc,Nr,Nl,Lur,Ldl,Ldr) (34) 
Lur+Th < min(Hc,Vc,Nr,Nl,Lul,Ldl,Ldr) (35) 
Ldl+Th < min(Hc,Vc,Nr,Nl,Lul,Lur,Ldr) (36) 
Ldr+Th < min(Hc,Vc,Nr,Nl,Lul,Lur,Ldl) (37) 
where Th is a threshold (a specific constant), and min is a 
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function which takes the minimum value among the elements in the 
parentheses. The correlation direction is the vertical 
direction when the expression (30) holds, the horizontal 
direction when the expression (31) holds, the diagonal direction 
toward lower right when the expression (32) holds, the diagonal 
direction toward lower left when the expression (33) holds, the 
upper left L-shaped direction when the expression (34) holds, 
the upper right L-shaped direction when the expression (35) 
holds, the lower left L-shaped direction when the expression 

(36) holds, and the lower right L-shaped direction when the 
expression (37) holds. When none of these expressions (30) to 

(37) holds, there is no correlation direction. 

Next, the interpolation process will be described. 

While in the third embodiment the interpolation process is 
described for the case where the correlation direction is the 
vertical or horizontal direction and the case where there is no 
correlation direction, in this sixth embodiment the 
interpolation process in the case where the correlation 
direction is any of the above-mentioned diagonal directions and 
L-shaped directions will be added. 

When the correlation direction is the diagonal direction 
toward the lower right, a shortage component RCy is calculated 
using only the peripheral pixels in the diagonal direction with 
respect to the target pixel Cyn, according to the expression 
(14). When the correlation direction is the diagonal direction 
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toward the lower left, a shortage component RCy is calculated 
using only the peripheral pixels in the diagonal direction with 
respect to the target pixel Cyn, according to the expression 
(15). When the correlation direction is the upper left L-shaped 
direction, a shortage component RCy is calculated using only the 
peripheral pixels in the upper left L-shaped direction with 
respect to the target pixel Cyn, according to the expression 
(26). When the correlation direction is the upper right L- 
shaped direction, a shortage component RCy is calculated using 
only the peripheral pixels in the upper right L-shaped direction 
with respect to the target pixel Cyn, according to the 
expression (27). When the correlation direction is the lower 
left L-shaped direction, a shortage component RCy is calculated 
using only the peripheral pixels in the lower left L-shaped 
direction with respect to the target pixel Cyn, according to the 
expression (28). When the correlation direction is the lower 
right L-shaped direction, a shortage component RCy is calculated 
using only the peripheral pixels in the lower right L-shaped 
direction with respect to the target pixel Cyn, according to the 
expression (29 ) . 

Thereafter, interpolation is carried out in like manner as 
described for the third embodiment, whereby all of the luminance 
signals are obtained. 

According to the sixth embodiment, as described above, not 
only the correlations in the vertical and horizontal directions 



but also the correlations in the respective diagonal directions 
and L-shaped directions are detected to be used for 
interpolation. Therefore, degradation of resolution is 
suppressed in the diagonal directions and the L-shaped 
directions as well as in the vertical and horizontal directions. 
Embodiment: 7 . 

Next, a seventh embodiment corresponding to Claim 10 of 
the present invention will be described. 

The construction of this seventh embodiment is identical 
to those of the third to seventh embodiments except the 
correlation calculating method by the correlation calculation 
circuit 6. in the third to sixth embodiments, the correlation 
vc in the vertical direction, the correlation He in the 
horizontal direction, the correlation Nr in the diagonal 
direction toward the lower right, the correlation Nl in the 
diagonal direction toward the lower left, the correlation Lul in 
the upper left L-shaped direction, the correlation Lur in the 
upper right L-shaped direction, the correlation Ldl in the lower 
left L-shaped direction, and the correlation Ldr in the lower 
right L-shaped direction are obtained by operation using the 
pixels of the same color. In this seventh embodiment, however, 
these correlations are obtained by operation using adjacent 
pixels of different colors, according to the following 
expressions . 

Vc=|Wu-Cyn|+|wd-Cyn| (38) 
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Hc= j Wl-Cyn | + 1 Wr-Cyn | 



(39) 



Nr=| (Wu+Wl)/2-Cyn|+| ( Wd+Wr ) / 2 -Cyn | 



(40) 



Nl= | ( Wu+Wr ) /2 -Cyn | + | ( Wd+Wl ) / 2-Cyn | 



(41) 



Lul= | Wu-Cyn | + | Wl-Cyn | 



(42) 



Ldl= | Wd-Cyn | + | Wl-Cyn | 



(43) 



Lur= | Wu-Cyn j + j Wr-Cyn | 
Ldr= | Wd-Cyn | + | Wr-Cyn j 



(44) 



(45) 



Decision on the correlation direction and interpolation 
are carried out in like manner as those mentioned for the third 
to sixth embodiments. 

According to the seventh embodiment, since the 
correlations are obtained by operation between adjacent pixels 
of different colors, the precision in calculating the 
correlations is improved. 
Embodiment 8. 

Next, an eighth embodiment according to Claim 11 of the 
present invention will be described. 

The construction of this eighth embodiment is identical to 
those of the third to sixth embodiments except the interpolation 
method by the interpolation circuit 7. 

That is, in this eighth embodiment, when the target pixel 
Cyn obtained in any of the third to sixth embodiments is 
subjected to interpolation, the luminance W' of this target 
pixel Cyn is obtained using only the peripheral W pixels without 
using the pixel Cyn itself, according to the following 



expressions . 

When the correlation direction is the vertical direction, 

W'=(Wu+Wd)/2 (46) 
When the correlation direction is the horizontal direction, 

W'=(Wl+Wr)/2 (47) 
When the correlation direction is the upper left L-shaped 
direction, 

W'=(Wu+Wl)/2 (48) 
When the correlation direction is the lower left L-shaped 
direction, 

W'=(Wd+Wl)/2 (49) 
When the correlation direction is the upper right L-shaped 
direction , 

W'=(Wu+Wr)/2 (50) 
When the correlation direction is the lower right L-shaped 
direction, 

W'=(Wd+Wr)/2 (51) 
When the correlation direction is other than those 
mentioned above, 

W'=(Wu+Wd+Wl+Wr)/4 (52) 
According to this eighth embodiment, since the luminance 
signals are calculated using only the W pixels, the 
interpolation precision is improved, resulting in a high- 
resolution image without luminance variations. 
Embodiment 9. 



Next, a ninth embodiment corresponding to Claims 13 and 14 
of the present invention will be described with reference to 
figures 10 and 11. 

In this ninth embodiment, when a target pixel to be 
interpolated is decided as having a correlation in a specific 
direction in the correlation detecting process according to any 
of the aforementioned embodiments, the color-difference signal 
(R-Y,B-Y) in the position of the target pixel is multiplied by a 
gain not larger than 1, regardless of the degree of the 
correlation. 

Figure 10 shows the relationship between the degree of 
correlation in the direction which is decided as having the 
strongest correlation, and the gain by which the color- 
difference signal is to be multiplied. Although a pseudo 
chrominance signal tends to appear at the luminance edge, the 
pseudo chrominance can be suppressed when the interpolation 
circuit 7 has the function of multiplying the color-difference 
signal by a gain not larger than 1. Further, since the 
correlation detection circuit 6 may have the function of 
detecting the edge, the pseudo chrominance can be suppressed 
without providing a special luminance-edge detection circuit. 

Further, the gain by which the color-difference signal is 
multiplied may be varied according to the degree of correlation 
in the direction decided as having the strongest correlation. 
Figure 11 shows an example of relationship between the degree of 
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correlation in the direction decided as having the strongest 
correlation and the gain by which the color-difference signal is 
multiplied. In this case, however, the stronger the correlation 
is, the smaller the degree of correlation is. 

in figure 11, a width Thl is given to the correlation, and 
the gain by which the color-difference signal is multiplied is 
gradually decreased for each width Thl. Generally, the larger 
the difference in luminance at the edge is, the thicker the 
pseudo chrominance becomes. That is, since the possibility of 
generating thick pseudo chrominance is higher as the correlation 
is smaller, the level of the color-difference signal can be 
reduced according to the possibility, whereby the pseudo 
chrominance can be suppressed effectively. 
Embodiment 10. 

Hereinafter, a tenth embodiment corresponding to Claims 15 
and 16 of the present invention will be described with reference 
to figures 12, 13, 14, and 15. In these figures, the same 
reference numerals as those used for the aforementioned 
embodiments denote the same or corresponding parts. 

Figure 12 shows the structure of a solid-state color image 
pickup apparatus according to the tenth embodiment of the 
invention, and this is fundamentally identical to that shown in 
figure 6 except that a frequency characteristic adjustment 
circuit 10 is added to the structure of figure 6. The 
interpolation circuit 7 shown in figure 6 is separated into a 
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luminance signal interpolation circuit 8 and a color-difference 
signal interpolation circuit 9 in figure 12, and the frequency 
characteristic adjustment circuit 10 is inserted just before the 
color-difference signal interpolation circuit 9. The frequency 
characteristic adjustment circuit 10 adjusts the frequency of 
the image signal stored in the storage circuit 5, and outputs it 
to the color-difference signal interpolation circuit 9. 

Figure 13 is a schematic diagram for explaining the 
operation of the frequency characteristic adjustment circuit 10 
for restricting the frequency band to the low-pass band, and 
adjusting the frequency characteristic of the color-difference 
signal. The frequency characteristic adjustment using a low- 
pass filter can be performed in the vertical direction, the 
horizontal direction, the vertical and horizontal directions, 
and the diagonal direction. When using the low-pass filter in 
-che vertical direction, 2n+l (n=l,2,...) signals of the same 
color in the vertical direction with a target pixel (a pixel 
whose frequency characteristic is to be adjusted) in the center 
are used, and each signal is multiplied by a factor for deciding 
the filter characteristic, and the average of these signals is 
calculated. For example, in figure 13, when three signals of 
the same color in the vertical direction with the Cy23 as the 
target pixel are filtered by the low-pass filter, the output 
signal Cy23 * from the frequency characteristic adjustment 
circuit 10 is represented" as follows. 
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Cy23 '=(Cy03+Cy23+Cy43)/3 (53) 
Likewise, when using the low-pass filter in the horizontal 
direction, 2n+l (n=l,2,...) signals of the same color in the 
horizontal direction with a target pixel (a pixel whose 
frequency characteristic is to be adjusted) in the center are 
used, and each signal is multiplied by a factor for deciding the 
filter characteristic, and the average of these signals is 
calculated. For example, in figure 13, when three signals of 
the same color in the horizontal direction with the Cy23 as the 
target pixel are filtered by the low-pass filter with all of the 
factors being 1, the output signal Cy23 ' from the frequency 
characteristic adjustment circuit 10 is represented as follows. 

Cy23'=(Cy21+Cy23+Cy25)/3 (54) 
Likewise, when using the low-pass filter in the vertical 
and horizontal directions, ( 2n+l ) X (2m+l ) (n,m=l , 2 , . . . ) signals 
of the same color in the vertical and horizontal directions with 
a target pixel (a pixel whose frequency characteristic is to be 
adjusted) in the center are used, and each signal is multiplied 
by a factor for deciding the filter characteristic, and the 
average of these signals is calculated. For example, in figure 
13, when nine signals of the same color in the vertical and 
horizontal directions with the Cy23 as the target pixel are 
filtered by the low-pass filter with all of the factors being 1 , 
the resultant signal is represented as follows. 

Cy23 '=(Cy01+Cy03+Cy05+Cy21+Cy23+Cy25+Cy41+Cy43+Cy45)/9 (55) 



Likewise, when using the low-pass filter in the diagonal 
directions, 2n+2m+l (n,m=l,2, . . . ) signals of the same color in 
the vertical and horizontal cross-shaped directions with a 
target pixel (a pixel whose frequency characteristic is to be 
adjusted) in the center are used, and each signal is multiplied 
by a factor for deciding the filter characteristic, and the 
average of these signals is calculated. For example, in figure 
13, when five signals of the same color in the vertical and 
horizontal cross-shaped directions with the Cy23 as the target 
pixel are filtered by the low-pass filter with all of the 
factors being 1, the resultant signal is represented as follows. 
Cy23 '=(Cy03+Cy21+Cy23+Cy25+Cy43 )/9 (56) 
The above-described operation for adjusting the frequency 
characteristic is performed on all of the chrominance signals 
which are required for compositing the color-difference signal 
by int er po 1 at ion . 

For example, when compositing the color-difference signal 
using the Cy23 as the target pixel and the W22 and W24 as the 
peripheral pixels, in order to apply the low-pass filter in the 
vertical and horizontal directions, the chrominance signals 
required for interpolation are calculated as shown in the 
following expressions (57), (58), and (59). 

W22 '=(W00+W02+W04+W20+W22+W24+W40+W42+W44 )/9 (57) 

Cy23 '=(Cy01+Cy03+Cy05+Cy21+Cy23+Cy25+Cy41+Cy43+Cy45) /9 (58) 

W24' = (W02+W04 +W0 6+W2 0 +W2 2 +W2 4 +W2 6 +W4 2 +W4 4 +W4 6 ) / 9 (59) 



60 



In the color-difference signal interpolation circuit 9 
shown in figure 12, the R-Y color-difference signal is 
calculated using the Cy23', W22 ' , and W24 ' according to the 
following expression. 

R-Y=AX (W22 '+W24 ' )-BXCy23 ' (60) 
wherein A and B are constants depending on the white balance or 
the like. Further when the Ye pixels are subjected to the 
above-described frequency characteristic adjustment and color- 
difference signal compensation using the same positional 
relationship as described above, the B-Y color-difference signal 
can be obtained. 

Figure 14 shows the amplitude characteristics 11 using 
three-point averaging as a method of frequency characteristic 
adjustment for the chrominance signal, and the amplitude 
characteristics 12 using linear interpolation as a method of 
interpolation for the color-difference signal, and these 
characteristics combined are shown by the solid line. The 
abscissa shows the frequency, and the sampling frequency of each 
chrominance signal is indicated by K . As shown in figure 14, 
when linear interpolation is performed using the chrominance 
signal subjected to frequency characteristic adjustment, the 
frequency component including aliasing, which appears in the 
vicinity of 7T/2 of the chrominance signal (shown by the dashed 
line in figure 15), is reduced. 

In this tenth embodiment, when the chrominance signal 
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stored in the storage circuit 5 contains high frequency 
components, the frequency characteristic adjustment circuit 10 
reduces the frequency component including aliasing, and the 
color-difference signal interpolation circuit 9 composites the 
color-difference signal by interpolation using the chrominance 
signal whose frequency characteristic is adjusted, whereby the 
pseudo chrominance signal is reduced. 
Embodiment 11. 

Hereinafter, an eleventh embodiment corresponding to 
Claims 17 and 18 of the present invention will be described with 
reference to figure 16. in figure 16, the same reference 
numerals as those used for the aforementioned embodiments denote 
the same constituents . 

Figure 16 shows the structure of a solid-state color image 
pickup apparatus according to the eleventh embodiment, and it is 
fundamentally identical to the structure shown in figure 12 
except that the frequency characteristic adjustment circuit 10 
is controlled by the output from the correlation detection 
circuit 6. 

In this construction, when the correlation detection 
circuit 6 decides that there is a correlation direction, the 
frequency characteristic of the chrominance signal of a target 
pixel to be interpolated is adjusted by the frequency 
characteristic adjustment circuit 10. Further, using the color 
signal of the target pixel whose frequency characteristic is 
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adjusted, the color-difference signal is calculated by the 
color-difference signal interpolation circuit 9. This process 
is identical to that mentioned for the tenth embodiment. 

On the other hand, when the correlation detection circuit 
6 decides that there is no correlation direction, the 
chrominance signal of the target pixel is sent to the color- 
difference signal interpolation circuit 9 without being 
processed in the frequency characteristic adjustment circuit 10. 
In the interpolation circuit 9, the color-difference signal is 
calculated using the chrominance signal and the luminance signal 
from the luminance signal interpolation circuit 8. 

As described above, in this eleventh embodiment, the 
frequency characteristic of the chrominance signal including 
high-frequency components and having the correlation direction 
is adjusted by the frequency characteristic adjustment circuit 
10, whereby occurrence of pseudo chrominance signals is reduced. 
On the other hand, since the chrominance signal having no 
correlation direction does not include pseudo chrominance 
components inherently, this chrominance signal need not be 
subjected to adjustment of frequency characteristic, and so the 
frequency components of the chrominance signal are not 
attenuated by adjustment of frequency characteristic, whereby 
the color reproducibility is maintained. 



APPLICABILITY IN INDUSTORY 
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As described above, in the solid-state color image pickup 
apparatus according to the present invention, four pieces of 
luminance data and two pieces of chrominance data are taken out 
of vertically and horizontally adjacent 4 pixels of the color 
separation filter placed on the surface of the solid-state image 
pickup device, whereby the luminance resolution is improved, and 
degradation of chrominance resolution is reduced. Therefore, 
this is valuable as a signal processing method of the solid- 
state color image pickup apparatus which obtains high resolution 
by performing interpolation between pixels. 
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CLAIMS 

1. A solid-state color image pickup apparatus comprising: 
a solid-state image pickup device being provided with; 
a color separation filter having one arrangement 
pattern comprising four pixels adjoining vertically and 
horizontally, said color separation filter of this arrangement 
pattern comprising two whole-color-pass filters corresponding to 
two pixels, a cyan-pass filter corresponding to one pixel, and a 
yellow-pass filter corresponding to one pixel, and repeating the 
arrangement pattern in the vertical and horizontal directions, 
and 

means for separately taking data corresponding to each 
pixel of the color separation filter, and 

a signal processing circuit for taking four luminance 
signals and two kinds of color-difference signals, for one of 
the arrangement patterns, from each pixel data taken out of the 
solid-state image pickup device; at this time, said signal 
processing circuit generating two of the four luminance signals 
by using only the data from the whole-color-pass filters, and 
generating the remaining two luminance signals by using the data 
from the whole-color-pass filters and the data of peripheral 
pixels in the vicinity of the four pixels adjoining vertically 
and horizontally, and generating the two kinds of color- 
difference signals by using the data from the cyan or yellow- 
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pass filter and the data from the peripheral pixels. 

2. A solid-state color image pickup apparatus as defined in 
Claim 1, wherein said color separation filter having one 
arrangement pattern comprising vertically and horizontally 
adjoining four pixels is constituted such that the four pixels 
are vertical two pixels X horizontal two pixels, and six 
signals comprising four luminance signals and two kinds of 
color-difference signals are generated from the data taken out 
of this arrangement pattern, and the six signals so generated 
are output to a device of 4:2:0 format. 

3. A solid-state color image pickup apparatus as defined in 
Claim 1, wherein said color separation filter having one 
arrangement pattern comprising vertically and horizontally 
adjoining four pixels is constituted such that the four pixels 
are vertical one pixel X horizontal four pixels, and six 
signals comprising four luminance signals and two kinds of 
color-difference signals are generated from the data taken out 
of the arrangement pattern, and the six signals so generated are 
output to a device of 4:1:1 format. 

4. A solid-state color image pickup apparatus as defined in 
Claim 1: 

wherein said color separation filter having one arrangement 
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pattern comprising vertically and horizontally adjoining four 
pixels, comprises a whole-color-pass filter and a cyan-pass 
filter for the upper two pixels from the left, and a yellow-pass 
filter and a whole-color-pass filter for the lower two pixels 
from the left; and 

said apparatus further comprising: 

storage means for capturing a chrominance signal outputted 
from each pixel of the solid-state image pickup device, and 
storing it; 

correlation calculation means for calculating the 
correlation between a target pixel to be interpolated and plural 
pixels in the vicinity of the target pixel, said target pixel 
being any of a cyan signal pixel and a yellow signal pixel 
stored in the storage means; and 

interpolation means for interpolating the target pixel in a 
direction along which the calculated correlation is relatively 
large, and calculating a whole-color-pass signal in the position 
of the target pixel. 

5. A solid-state color image pickup apparatus as defined in 
Claim 4, wherein said correlation calculation means calculates 
the correlation between the target pixel and the pixels in the 
vicinity of the target pixel, including the target pixel, in the 
horizontal or vertical direction. 
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6. A solid-state color image pickup apparatus as defined in 
Claim 4, wherein said correlation calculation means calculates 
the correlation between the target pixel and the pixels in the 
vicinity of the target pixel, including the target pixel, in the 
horizontal or vertical direction and, further, in the diagonal 
direction. 

7. A solid-state color image pickup apparatus as defined in 
Claim 4, wherein said correlation calculation means calculates 
the correlation between the target pixel and the pixels in the 
vicinity of the target pixel, including the target pixel, in the 
horizontal or vertical direction and, further, in the upper 
right direction, or lower right direction, or upper left 
direction, or lower left direction. 

8. A solid-state color image pickup apparatus as defined in 
Claim 4, wherein said correlation calculation means calculates 
the correlation between the target pixel and the pixels in the 
vicinity of the target pixel, including the target pixel, in the 
horizontal or vertical direction, and in the diagonal direction, 
and further, in the upper right direction, or lower right 
direction, or upper left direction, or lower left direction. 

9. A solid-state color image pickup apparatus as defined in 
Claim 4, wherein said correlation calculation means calculates 
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the correlation by performing operation on signals of the same 
color, between the target pixel and the pixels in the vicinity 
of the target pixel. 

10. A solid-state color image pickup apparatus as defined in 
Claim 4, wherein said correlation calculation means calculates 
the correlation by performing operation on adjacent pixels being 
signals of different colors, between the target pixel and the 
pixels in the vicinity of the target pixel. 

11. A solid-state color image pickup apparatus as defined in 
Claim 4, wherein said interpolation means performs interpolation 
using only signals of the same color as that of a chrominance 
signal to be generated, in the vicinity of the target pixel, 
without using the chrominance signal of the target pixel in the 
direction where the correlation calculated by the correlation 
calculation means is relatively large. 

12. A solid-state color image pickup apparatus as defined in 
Claim 4, wherein said interpolation means calculates a shortage 
of a chrominance signal to be generated, from the pixels in the 
vicinity of the target pixel, using the chrominance signal of 
the target pixel in the direction where the correlation 
calculated by the correlation calculation means is relatively 
large, and performs interpolation on the shortage. 
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13. A solid-state color image pickup apparatus as defined in 
any of Claims 5 to 10 wherein, when the correlation calculated 
by the correlation calculation means is smaller than a given 
threshold, said interpolation means reduces the gain of the 
color-difference signal corresponding to the pixel. 

14. A solid-state color image pickup apparatus as defined in 
any of Claims 5 to 10 wherein, when the correlation calculated 
by the correlation calculation means is smaller than a given 
threshold, said interpolation means reduces the gain of the 
color-difference signal corresponding to the pixel, stepwise, 
according to the correlation. 

15. A solid-state color image pickup apparatus as defined in 
Claim 4, wherein said interpolation means is provided with 
frequency characteristic adjustment means for adjusting the 
frequency characteristic of each chrominance signal outputted 
from the solid-state image pickup device, and the interpolation 
means interpolates and composites a color-difference signal by 
using the chrominance signal whose frequency characteristic is 
adjusted. 

16. A solid-state color image pickup apparatus as defined in 
Claim 5, wherein said interpolation means is provided with 
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frequency characteristic adjustment means for adjusting the 
frequency characteristic of each chrominance signal outputted 
from the solid-state image pickup device, and the interpolation 
means interpolates and composites an R-Y color-difference signal 
in the position of the cyan-pass filter, and a B-Y color- 
difference signal in the position of the yellow-pass filter, by 
using the chrominance signal whose frequency characteristic is 
adjusted. 

17. A solid-state color image pickup apparatus as defined in 
Claim 15, wherein said interpolation means decides the 
correlation direction according to the correlation calculated by 
the correlation calculation means, performs frequency 
characteristic adjustment when there is a direction where the 
correlation is relatively large, and does not perform frequency 
characteristic adjustment when there is no direction where the 
correlation is relatively large. 

18. A solid-state color image pickup apparatus as defined in 
Claim 16, wherein said interpolation means decides the 
correlation direction according to the correlation calculated by 
the correlation calculation means , performs frequency 
characteristic adjustment when there is a direction where the 
correlation is relatively large, and does not perform frequency 
characteristic adjustment when there is no direction where the 
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correlation is relatively large. 



ABSTRACT 



A solid-state color image pickup apparatus comprises a 
solid-state image pickup device 2 which is provided with a color 
separation filter having one arrangement pattern comprising four 
pixels adjoining vertically and horizontally, and comprising a 
whole-color-pass filter and a cyan-pass filter for the upper two 
pixels from the left, and a yellow-pass filter and a whole- 
color-pass filter for the lower two pixels from the left, and 
the pattern being repeated; a storage means 5 for capturing a 
chrominance signal outputted from each pixel and storing it; a 
correlation calculation means 6 for calculating the correlation 
between a target pixel to be interpolated, which is a cyan 
signal pixel or a yellow signal pixel stored in the storage 
means, and plural pixels in the vicinity of the target pixel; 
and an interpolation means 7 for performing interpolation in a 
direction where the correlation is relatively large, and 
calculating a whole-color-pass signal in the position of the 
target pixel. 
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As a below named inventor, I hereby declare that: my residence, post office address and citizenship are as stated below next 
to my name; that I verily believe that I am the original, first and sole inventor (if only one name is listed below) or an original, first and 
joint inventor (if plural inventors are named below) of the subject matter which is claimed and for which a patent is sought on the 
invention entitled: 



Title: SOLID-STATE COLOR IMAGE PICKUP APPARATUS 



of which is described and claimed in: 
0 the attached specification, or 

0 the specification in application Serial No. , filed , and with amendments through (if applicable), 

or 

(X) the specification in International Application No. PCT/JP99/02385 . filed May 7, 1999 , and as amended on November 8, 2000 
(if applicable). 



I hereby state that I have reviewed and understand the content of the above-identified specification, including the claims, as amended 
by any amendment(s) referred to above. 



% acknowledge my duty to disclose to the Patent and Trademark Office all information known to me to be material to patentability as 
fdefined in Title 37, Code of Federal Regulations, §1.56. 



■A hereby claim priority benefits under Title 35, United States Code, §119 (and §172 if this application is for a Design) of any 
•^application^) for patent or inventor's certificate listed below and have also identified below any application for patent or inventor's 
^certificate having a filing date before that of the application on which priority is claimed: 



COUNTRY 
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DATE OF FILING 


PRIORITY 
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Hei. 10-125677 


May 8, 1998 
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Japan 


Hei. 11-67003 


March 12, 1999 
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I hereby claim the benefit under Title 35, United States Code §120 of any United States application(s) listed below and, insofar as the 
subject matter of each of the claims of this application is not disclosed in the prior United States application in the manner provided by 
the first paragraph of Title 35, United States Code §112, I acknowledge the duty to disclose information material to patentability as 
defined" in Title 37, Code of Federal Regulations, §1.56 which occurred between the filing date of the prior application and the national 
or PCT international filing date of this application: 



APPLICATION SERIAL NO. 


U.S. FILING DATE 


STATUS: PATENTED, PENDING, 
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And I hereby appoint Michael R. Davis, Reg. No.,25,134;. Matthew M. Jacob, Reg. No^OSylS^ Jeffrey Nolton, Reg. No, 
Warren M. Cheek, Jr., Reg. No. aa^STj-^ils Pedersen, Reg. No . 33.145: a nd Charles R. Watts, Reg. No- I -33rM3 T 'Who together 
constitute the firm of WENDEROTH, LIND & PONACK, L.L.P. , jointly and severally, attorneys to prosecute this application and to 
transact all business in the U.S. Patent and Trademark Office connected therewith. 



I hereby authorize the U.S. attorneys named herein to accept and follow instructions from HAYASE &CO. as to any action to be taken 
in the U .S. Patent and Trademark Office regarding this application without direct communication between the U .S. attorneys and myself. 
In the event of a change in the persons from whom instructions may be taken, the U .S. attorneys named herein will be so notified by 
me. 
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2033jCJJU£6lJ^_W rr -Sttit® 800 Area Code (202) 721-8200 

Washington, D.C. 20006 
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Area Code (202) 721-8250 
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FAMILY NAME FIRST GIVEN NAME SECOND CIVEIN NAME 

MIZUKI Hjromasa_ 


Residence & 
Citizenship 


CITY ^__-Y/^-«ATE OR COUNTRY COUNTRY OF CITIZENSHIP 

Saiiyo-_shj J^h Alapan Japan 


Post Office 
LAddress 


ADDRESS CITY STATE OR COUNTRY ZIP CODE 

339-14, Furukawa Kpu, Saijyo-shi, Ehime 793-0044 Japan 
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Second Inventor 


FAMILY NAME '-^"^ ^^nRST GIVEN NAME SECOND GIVEN NAME 

YAMASFJJTA JVlasaaki 


Residence & 
Citizenship 
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CITY /J-^TATE OR COUNTRY COUNTRY OF CITIZENSHIP 
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Post Office 
Address 


ADDRESS CITY STATE OR COUNTRY ZIP CODE 

1620-23, Hagyu, Niihama-shi, Ehime 792-0050 Japan 


Full Name of 
Third Inventor 


FAMILY NAME "^>.-C. <__--' FIRST GIVEN NAME SECOND GIVEN NAME 

KIYOSfflGE Yasushi 


i Residence & 
Citizenship 


" r ./-x / 

CITY y\ATATE OR COUNTRY COUNTRY OF CITIZENSHIP 

Saijyo-shi, \\ Japan Japan 


i Post Office 
1 Address 


ADDRESS ' CITY STATE OR COUNTRY ZIP CODE 

200, Fukutake Kou, Saijyo-shi, Ehime 793-0035 Japan 


| Full Name of 
1 Fourth Inventor 
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FAMILY Vt&afET " ^-C^ FIRST GIVEN NAME SECOND GIVEN NAME 

WATANABE Hidehiko._ 


! Residence & 
Citizenship 


CITY .^LJj\SrtfATt. OR COUNTRY COUNTRY OF CITIZENSHIP 
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Post Office 
Address 


ADDRESS CITY STATE OR COUNTRY ZIP CODE 

200, Fukutake Kou, Saijyo-shi, Ehime 793-0035 Japan 
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Residence & 
Citizenship 
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-» *ThVundersigned hereby grant(s) the firm of WENDEROTH, LIND &PONACK, L.L.P., 2033KStreet, 
N.W., Suite 800, Washington, DC 20006, the power to insert on this assignment any further identification which 
may be necessary or desirable in order to comply with the rules of the United States Patent Office for recordation 
of this document. 

In witness whereof, executed by the undersigned on the date(s) opposite the undersigned name(s). 



Date 


Oi*n^ 22- , 2<?<«? 
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of 


Inventor 


Date 
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of 


Inventor 


Date 
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Inventor 


Date 


22, 2000 
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of 


Inventor 


Date 
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Inventor 


Date 
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Inventor 



Hiromasa MIZUKI 



Yasushi KIYOSnj-vi^ , / 
Hidehiko WATANABE 



(This assignment should preferably be acknowledged before a United States Consul. If not, then the execu- 
tion by the Inventor(s) should be witnessed by at least two witnesses who sign here.) 



Witness . 



ACKNOWLEDGEMENT 



This day of _ 

personally came the above-named 



to me personally known as the individual(s) who executed the foregoing assignment, who did acknowledge to me 
that he (they) executed the same of his (their) own free will for the purposes therein set forth. 

Official Signature 
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U.S. Application Serial No Filing Date November 8, 2000 

Applicant Reference Number P-20511-02 Atty Docket No. 2000^1 520A 

Title Of Invention SOLID-STATE COLOR IMAGE PICKUP APPARATUS 



